diff --git a/src/PaddleClas/docs/Makefile b/src/PaddleClas/docs/Makefile new file mode 100644 index 0000000..d0c3cbf --- /dev/null +++ b/src/PaddleClas/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source +BUILDDIR = build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/src/PaddleClas/docs/en/Makefile b/src/PaddleClas/docs/en/Makefile new file mode 100644 index 0000000..d4bb2cb --- /dev/null +++ b/src/PaddleClas/docs/en/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = . +BUILDDIR = _build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/src/PaddleClas/docs/en/advanced_tutorials/DataAugmentation_en.md b/src/PaddleClas/docs/en/advanced_tutorials/DataAugmentation_en.md new file mode 100644 index 0000000..e6613d9 --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/DataAugmentation_en.md @@ -0,0 +1,338 @@ +# Image Augmentation +--- + +Experiments about data augmentation will be introduced in detail in this section. If you want to quickly experience these methods, please refer to [**Quick start PaddleClas in 30 miniutes**](../../tutorials/quick_start_en.md), which based on CIFAR100 dataset. If you want to know the content of related algorithms, please refer to [Data Augmentation Algorithm Introduction](../algorithm_introduction/DataAugmentation_en.md). + + +## Catalogue + +- [1. Configurations](#1) + - [1.1 AutoAugment](#1.1) + - [1.2 RandAugment](#1.2) + - [1.3 TimmAutoAugment](#1.3) + - [1.4 Cutout](#1.4) + - [1.5 RandomErasing](#1.5) + - [1.6 HideAndSeek](#1.6) + - [1.7 GridMask](#1.7) + - [1.8 Mixup](#1.8) + - [1.9 Cutmix](#1.9) + - [1.10 Use Mixup and Cutmix at the same time](#1.10) +- [2. Start training](#2) +- [3. Matters needing attention](#3) +- [4. Experiments](#4) + + + +## Configurations + +Since hyperparameters differ from different augmentation methods. For better understanding, we list 8 augmentation configuration files in `configs/DataAugment` based on ResNet50. Users can train the model with `tools/run.sh`. The following are 3 of them. + + +### 1.1 AutoAugment + +The configuration of the data augmentation method of `AotoAugment` is as follows. `AutoAugment` is converted on the uint8 data format, so its processing should be placed before the normalization operation (`NormalizeImage`). + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - AutoAugment: + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + + +### 1.2 RandAugment + +The configuration of the data augmentation method of `RandAugment` is as follows, where the user needs to specify the parameters `num_layers` and `magnitude`, and the default values ​​are `2` and `5` respectively. `RandAugment` is converted on the uint8 data format, so its processing should be placed before the normalization operation (`NormalizeImage`). + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - RandAugment: + num_layers: 2 + magnitude: 5 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + + +### 1.3 TimmAutoAugment + +The configuration of the data augmentation method of `TimmAutoAugment` is as follows, in which the user needs to specify the parameters `config_str`, `interpolation`, and `img_size`. The default values ​​are `rand-m9-mstd0.5-inc1` and `bicubic. `, `224`. `TimmAutoAugment` is converted on the uint8 data format, so its processing should be placed before the normalization operation (`NormalizeImage`). + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - TimmAutoAugment: + config_str: rand-m9-mstd0.5-inc1 + interpolation: bicubic + img_size: 224 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + + +### 1.4 Cutout + +The configuration of the data augmentation method of `Cutout` is as follows, where the user needs to specify the parameters `n_holes` and `length`, and the default values are `1` and `112` respectively. Similar to other image cropping data augmentation methods, `Cutout` can operate on data in uint8 format, or on data after normalization (`NormalizeImage`).The demo here is operated after normalization. + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - Cutout: + n_holes: 1 + length: 112 +``` + + +### 1.5 RandomErasing + +The configuration of the image augmentation method of `RandomErasing` is as follows, where the user needs to specify the parameters `EPSILON`, `sl`, `sh`, `r1`, `attempt`, `use_log_aspect`, `mode`, and the default values They are `0.25`, `0.02`, `1.0/3.0`, `0.3`, `10`, `True`, and `pixel`. Similar to other image cropping data augmentation methods, `RandomErasing` can operate on data in uint8 format, or on data after normalization (`NormalizeImage`).The demo here is operated after normalization. + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - RandomErasing: + EPSILON: 0.25 + sl: 0.02 + sh: 1.0/3.0 + r1: 0.3 + attempt: 10 + use_log_aspect: True + mode: pixel +``` + + +### 1.6 HideAndSeek + +The configuration of the image augmentation method of `HideAndSeek` is as follows. Similar to other image cropping data augmentation methods, `HideAndSeek` can operate on data in uint8 format, or on data after normalization (`NormalizeImage`).The demo here is operated after normalization. + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - HideAndSeek: +``` + + +### 1.7 GridMask + +The configuration of the image augmentation method of `GridMask` is as follows, where the user needs to specify the parameters `d1`, `d2`, `rotate`, `ratio`, `mode`, and the default values are `96`, `224 respectively `, `1`, `0.5`, `0`. Similar to other image cropping data augmentation methods, `HideAndSeek` can operate on data in uint8 format, or on data after normalization (`GridMask`).The demo here is operated after normalization. + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - GridMask: + d1: 96 + d2: 224 + rotate: 1 + ratio: 0.5 + mode: 0 +``` + + +### 1.8 Mixup + +The configuration of the data augmentation method of `Mixup` is as follows, where the user needs to specify the parameter `alpha`, and the default value is `0.2`. Similar to other image mixing data augmentation methods, `Mixup` is to perform image mix on the data in each batch after the image is processed, and the mixed images and labels are put into the network for training, +so it operates after image data processing (image transformation, image cropping). + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + batch_transform_ops: + - MixupOperator: + alpha: 0.2 +``` + + +### 1.9 Cutmix + +The configuration of the image augmentation method of `Cutmix` is as follows, where the user needs to specify the parameter `alpha`, and the default value is `0.2`. Similar to other image mixing data augmentation methods, `Mixup` is to perform image mix on the data in each batch after the image is processed, and the mixed images and labels are put into the network for training, +so it operates after image data processing (image transformation, image cropping). + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + batch_transform_ops: + - CutmixOperator: + alpha: 0.2 +``` + + +### 1.10 Use Mixup and Cutmix at the same time + +The configuration for both `Mixup` and `Cutmix` is as follows, in which the user needs to specify an additional parameter `prob`, which controls the probability of different data enhancements, and the default is `0.5`. + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - OpSampler: + MixupOperator: + alpha: 0.8 + prob: 0.5 + CutmixOperator: + alpha: 1.0 + prob: 0.5 +``` + + +## 2. Start training + +After you configure the training environment, similar to training other classification tasks, you only need to replace the configuration file in `tools/train.sh` with the configuration file of the corresponding data augmentation method. + + +The contents of `train.sh` are as follows: + +```bash +python3 -m paddle.distributed.launch \ + --selected_gpus="0,1,2,3" \ + --log_dir=ResNet50_Cutout \ + tools/train.py \ + -c ./ppcls/configs/ImageNet/DataAugment/ResNet50_Cutout.yaml +``` + +Run `train.sh`: + +```bash +sh tools/train.sh +``` + + +## 3. Matters needing attention + +* In addition, because the label needs to be aliased when the image is aliased, the accuracy of the training data cannot be calculated. The training accuracy rate was not printed during the training process. + +* The training data is more difficult with data augmentation, so the training loss may be larger, the training set accuracy is relatively low, but it has better generalization ability, so the validation set accuracy is relatively higher. + +* After the use of data augmentation, the model may tend to be underfitting. It is recommended to reduce `l2_decay` for better performance on validation set. + +* hyperparameters exist in almost all agmenatation methods. Here we provide hyperparameters for ImageNet1k dataset. User may need to finetune the hyperparameters on specified dataset. More training tricks can be referred to [Tricks](../models_training/train_strategy_en.md). + + +> If this document is helpful to you, welcome to star our project: [https://github.com/PaddlePaddle/PaddleClas](https://github.com/PaddlePaddle/PaddleClas) + + +## 4. Experiments + +Based on PaddleClas, Metrics of different augmentation methods on ImageNet1k dataset are as follows. + + +| Model | Learning strategy | l2 decay | batch size | epoch | Augmentation method | Top1 Acc | Reference | +|-------------|------------------|--------------|------------|-------|----------------|------------|----| +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | Standard transform | 0.7731 | - | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | AutoAugment | 0.7795 | 0.7763 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | mixup | 0.7828 | 0.7790 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | cutmix | 0.7839 | 0.7860 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | cutout | 0.7801 | - | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | gridmask | 0.7785 | 0.7790 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | random-augment | 0.7770 | 0.7760 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | random erasing | 0.7791 | - | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | hide and seek | 0.7743 | 0.7720 | + + +**note**: +* In the experiment here, for better comparison, we fixed the l2 decay to 1e-4. To achieve higher accuracy, we recommend trying to use a smaller l2 decay. Combined with data augmentaton, we found that reducing l2 decay from 1e-4 to 7e-5 can bring at least 0.3~0.5% accuracy improvement. +* We have not yet combined different strategies or verified, whch is our future work. diff --git a/src/PaddleClas/docs/en/advanced_tutorials/code_overview_en.md b/src/PaddleClas/docs/en/advanced_tutorials/code_overview_en.md new file mode 100644 index 0000000..05f4b57 --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/code_overview_en.md @@ -0,0 +1,303 @@ +# Code Overview + +## Catalogue + +- [1. Overview of Code and Content](#1) +- [2. Training Module](#2) + - [2.1 Data](#2.1) + - [2.2 Model Structure](#2.2) + - [2.3 Loss Function](#2.3) + - [2.4 Optimizer, Learning Rate Decay, and Weight Decay](#2.4) + - [2.5 Evaluation During Training](#2.5) + - [2.6 Model Saving](#2.6) + - [2.7 Model Pruning and Quantification](#2.7) +- [3. Codes and Methods for Inference and Deployment](#3) + + + +## 1. Overview of Code and Content + +The main code and content structure of PaddleClas are as follows: + +- benchmark: shell scripts to test the speed metrics of different models in PaddleClas, such as single-card training speed metrics, multi-card training speed metrics, etc. +- dataset: datasets and the scripts used to process datasets. The scripts are responsible for processing the dataset into a suitable format for Dataloader. +- deploy: code for deployment, including deployment tools, which support python/cpp inference, Hub Serveing, Paddle Lite, Slim offline quantification and other deployment methods. +- ppcls: code for training and evaluation which is the main body of the PaddleClas framework. It also contains configuration files, and specific code of model training, evaluation, inference, dynamic to static export, etc. +- tools: entry functions and scripts for training, evaluation, inference, and dynamic to static export. +- The requirements.txt file is adopted to install the dependencies for PaddleClas. Use pip for upgrading, installation, and application. +- test_tipc: TIPC tests of PaddleClas models from training to prediction to verify that whether each function works properly. + + + +## 2. Training Module + +Modules of training deep learning model mainly contains data, model structure, loss function, +strategies such as optimizer, learning rate decay, and weight decay strategy, etc., which are explained below. + + + +### 2.1 Data + +For supervised tasks, the training data generally contains the raw data and its annotation. +In a single-label-based image classification task, the raw data refers to the image data, +while the annotation is the class to which the image data belongs. In PaddleClas, a label file, +in the following format, is required for training, +with each row containing one training sample and separated by a separator (space by default), +representing the image path and the class label respectively. + +``` +train/n01440764/n01440764_10026.JPEG 0 +train/n01440764/n01440764_10027.JPEG 0 +``` + +`ppcls/data/dataloader/common_dataset.py` contains the `CommonDataset` class inherited from `paddle.io.Dataset`, +which is a dataset class that can index and fetch a given sample by a key value. +Dataset classes such as `ImageNetDataset`, `LogoDataset`, `CommonDataset`, etc. are all inherited from this class. + +The raw image needs to be preprocessed before training. +The standard data preprocessing during training contains +`DecodeImage`, `RandCropImage`, `RandFlipImage`, `NormalizeImage`, and `ToCHWImage`. +The data preprocessing is mainly in the `transforms` field, which is presented in a list, +and then converts the data in order, as reflected in the configuration file below. + +```yaml +DataLoader: + Train: + dataset: + name: ImageNetDataset + image_root: ./dataset/ILSVRC2012/ + cls_label_path: ./dataset/ILSVRC2012/train_list.txt + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + +PaddleClas also contains `AutoAugment`, `RandAugment`, and other data augmentation methods, +which can also be configured in the configuration file and thus added to the data preprocessing of the training. +Each data augmentation and process method is implemented as a class for easy migration and reuse. +For more specific implementation of data processing, please refer to the code under `ppcls/data/preprocess/ops/`. + +You can also use methods such as mixup or cutmix to augment the data that make up a batch. +PaddleClas integrates `MixupOperator`, `CutmixOperator`, `FmixOperator`, and other batch-based data augmentation methods, +which can be configured by deploying the mix parameter in the configuration file. +For code implementation, please refer to `ppcls/data/preprocess /batch_ops/batch_operators.py`. + +In image classification, the data post-processing is mainly `argmax` operation, which is not elaborated here. + + + +### 2.2 Model Structure + +The model in the configuration file is structured as follows: + +```yaml +Arch: + name: ResNet50 + class_num: 1000 + pretrained: False + use_ssld: False +``` + +`Arch.name`: the name of the model + +`Arch.pretrained`: whether to add a pre-trained model + +`Arch.use_ssld`: whether to use a pre-trained model based on `SSLD` knowledge distillation. + +All model names are defined in `ppcls/arch/backbone/__init__.py`. + +Correspondingly, the model object is created in `ppcls/arch/__init__.py` with the `build_model` method. + +```python +def build_model(config): + config = copy.deepcopy(config) + model_type = config.pop("name") + mod = importlib.import_module(__name__) + arch = getattr(mod, model_type)(**config) + return arch +``` + + + +### 2.3 Loss Function + +PaddleClas implement `CELoss` , `JSDivLoss`, `TripletLoss`, `CenterLoss` and other loss functions, all defined in `ppcls/loss`. + +In the `ppcls/loss/__init__.py` file, `CombinedLoss` is used to construct and combine loss functions. +The loss functions and calculation methods required in different training strategies are disparate, +and the following factors are considered by PaddleClas in the construction of the loss function. + +1. whether to use label smooth +2. whether to use mixup or cutmix +3. whether to use distillation method for training +4. whether to train metric learning + +User can specify the type and weight of the loss function in the configuration file, +such as adding TripletLossV2 to the training, the configuration file is as follows: + +```yaml +Loss: + Train: + - CELoss: + weight: 1.0 + - TripletLossV2: + weight: 1.0 + margin: 0.5 +``` + + + +### 2.4 Optimizer, Learning Rate Decay, and Weight Decay + +In image classification tasks, `Momentum` is a commonly used optimizer, +and several optimizer strategies such as `Momentum`, `RMSProp`, `Adam`, and `AdamW` are provided in PaddleClas. + +The weight decay strategy is a common regularization method, mainly adopted to prevent model overfitting. +Two weight decay strategies, `L1Decay` and `L2Decay`, are provided in PaddleClas. + +Learning rate decay is an essential training method for accuracy improvement in image classification tasks. +PaddleClas currently supports `Cosine`, `Piecewise`, `Linear`, and other learning rate decay strategies. + +In the configuration file, the optimizer, weight decay, +and learning rate decay strategies can be configured with the following fields. + +```yaml +Optimizer: + name: Momentum + momentum: 0.9 + lr: + name: Piecewise + learning_rate: 0.1 + decay_epochs: [30, 60, 90] + values: [0.1, 0.01, 0.001, 0.0001] + regularizer: + name: 'L2' + coeff: 0.0001 +``` + +Employ `build_optimizer` in `ppcls/optimizer/__init__.py` to create the optimizer and learning rate objects. + +```python +def build_optimizer(config, epochs, step_each_epoch, parameters): + config = copy.deepcopy(config) + # step1 build lr + lr = build_lr_scheduler(config.pop('lr'), epochs, step_each_epoch) + logger.debug("build lr ({}) success..".format(lr)) + # step2 build regularization + if 'regularizer' in config and config['regularizer'] is not None: + reg_config = config.pop('regularizer') + reg_name = reg_config.pop('name') + 'Decay' + reg = getattr(paddle.regularizer, reg_name)(**reg_config) + else: + reg = None + logger.debug("build regularizer ({}) success..".format(reg)) + # step3 build optimizer + optim_name = config.pop('name') + if 'clip_norm' in config: + clip_norm = config.pop('clip_norm') + grad_clip = paddle.nn.ClipGradByNorm(clip_norm=clip_norm) + else: + grad_clip = None + optim = getattr(optimizer, optim_name)(learning_rate=lr, + weight_decay=reg, + grad_clip=grad_clip, + **config)(parameters=parameters) + logger.debug("build optimizer ({}) success..".format(optim)) + return optim, lr +``` + +Different optimizers and weight decay strategies are implemented as classes, +which can be found in the file `ppcls/optimizer/optimizer.py`. +Different learning rate decay strategies can be found in the file `ppcls/optimizer/learning_rate.py`. + + + +### 2.5 Evaluation During Training + +When training the model, you can set the interval of model saving, +or you can evaluate the validation set every several epochs so that the model with the best accuracy can be saved. +Follow the examples below to configure. + +``` +Global: + save_interval: 1 # epoch interval of model saving + eval_during_train: True # whether evaluate during training + eval_interval: 1 # epoch interval of evaluation +``` + + + +### 2.6 Model Saving + +The model is saved through the `paddle.save()` function of the Paddle framework. +The dynamic graph version of the model is saved in the form of a dictionary to facilitate further training. +The specific implementation is as follows: + +```python +def save_model(program, model_path, epoch_id, prefix='ppcls'): + model_path = os.path.join(model_path, str(epoch_id)) + _mkdir_if_not_exist(model_path) + model_prefix = os.path.join(model_path, prefix) + paddle.static.save(program, model_prefix) + logger.info( + logger.coloring("Already save model in {}".format(model_path), "HEADER")) +``` + +When saving, there are two things to keep in mind: + +1. Only save the model on node 0, otherwise, if all nodes save models to the same path, +a file conflict may occur during multi-card training when multiple nodes write files, +preventing the final saved model from being loaded correctly. +2. Optimizer parameters also need to be saved to facilitate subsequent loading of breakpoints for training. + + + +### 2.7 Model Pruning and Quantification + +If you want to conduct compression training, please configure with the following fields. + +1. Model pruning： + +```yaml +Slim: + prune: + name: fpgm + pruned_ratio: 0.3 +``` + +2. Model quantification： + +```yaml +Slim: + quant: + name: pact +``` +For details of the training method, see [Pruning and Quantification Application](model_prune_quantization_en.md), +and the algorithm is described in [Pruning and Quantification algorithms](model_prune_quantization_en.md). + + + +## 3. Codes and Methods for Inference and Deployment + +- If you wish to quantify the classification model offline, please refer to +[Model Pruning and Quantification Tutorial](model_prune_quantization_en.md) for offline quantification. +- If you wish to use python for server-side deployment, +please refer to [Python Inference Tutorial](../inference_deployment/python_deploy_en.md). +- If you wish to use cpp for server-side deployment, +please refer to [Cpp Inference Tutorial](../inference_deployment/cpp_deploy_en.md). +- If you wish to deploy the classification model as a service, +please refer to the [Hub Serving Inference Deployment Tutorial](../inference_deployment/paddle_hub_serving_deploy_en.md). +- If you wish to use classification models for inference on mobile, +please refer to [PaddleLite Inference Deployment Tutorial](../inference_deployment/paddle_lite_deploy_en.md) +- If you wish to use the whl package for inference of classification models, +please refer to [whl Package Inference](../inference_deployment/whl_deploy_en.md) . diff --git a/src/PaddleClas/docs/en/advanced_tutorials/distillation/distillation_en.md b/src/PaddleClas/docs/en/advanced_tutorials/distillation/distillation_en.md new file mode 100644 index 0000000..f1d39a3 --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/distillation/distillation_en.md @@ -0,0 +1,245 @@ + +# Knowledge Distillation + +## Introduction of model compression methods + +In recent years, deep neural networks have been proven to be an extremely effective method to solve problems in the fields of computer vision and natural language processing. The deep learning methods performs better than traditional methods with suitable network structure and training process. + +With enough training data, increasing parameters of the neural network by building a reasonabe network can significantly the model performance. But this increases the model complexity, which takes too much computation cost in real scenarios. + + +Parameter redundancy exists in deep neural networks. There are several methods to compress the model such as pruning ,quantization, knowledge distillation, etc. Knowledge distillation refers to using the teacher model to guide the student model to learn specific tasks, ensuring that the small model has a relatively large effect improvement with the computation cost unchanged, and even obtains similar accuracy with the large model [1]. Combining some of the existing distillation methods [2,3], PaddleClas provides a simple semi-supervised label knowledge distillation solution (SSLD). Top-1 Accuarcy on ImageNet1k dataset has an improvement of more than 3% based on ResNet_vd and MobileNet series, which can be shown as below. + + +![](../../../images/distillation/distillation_perform_s.jpg) + + +## SSLD + +### Introduction + +The following figure shows the framework of SSLD. + +![](../../../images/distillation/ppcls_distillation.png) + +First, we select nearly 4 million images from ImageNet22k dataset, and integrate it with the ImageNet-1k training set to get a new dataset containing 5 million images. Then, we combine the student model and the teacher model into a new network, which outputs the predictions of the student model and the teacher model, respectively. The gradient of the entire network of the teacher model is fixed. Finally, we use JS divergence loss as the loss function for the training process. Here we take MobileNetV3 distillation task as an example, and introduce key points of SSLD. + +* Choice of the teacher model, During knowledge distillation, it may not be an optimal solution if the structure of the teacher model and the student model are too different. Under the same structure, the teacher model with higher accuracy leads to better performance for the student model during distillation. Compared with the 79.12% ResNet50_vd teacher model, using the 82.4% teacher model can bring a 0.4% accuracy improvement on Top-1 accuracy (`75.6%-> 76.0%`). + +* Improvement of loss function. The most commonly used loss function for classification is cross entropy loss. We find that when using soft label for training, KL divergence loss is almost useless to improve model performance compared to cross entropy loss, but The accuracy has a 0.2% improvement using JS divergence loss (`76.0%-> 76.2%`). Loss function in SSLD is JS divergence loss. + +* More iteration number. It is only 120 for the baseline experiment. We can achieve a 0.9% improvement by setting it as 360 (`76.2%-> 77.1%`). + +* There is not need for laleled data in SSLD, which leads to convenient training data expansion. label is not utilized when computing the loss function, therefore the unlabeled data can also be used to train the network. The label-free distillation strategy of this distillation solution has also greatly improved the upper performance limit of student models (`77.1%-> 78.5%`). + +* ImageNet1k finetune. ImageNet1k training set is used for finetuning, which brings a 0.4% accuracy improvement (`78.5%-> 78.9%`). + + +### Data selection + +* An important feature of the SSLD distillation scheme is no need for labeled images, so the dataset size can be arbitrarily expanded. Considering the limitation of computing resources, we here only expand the training set of the distillation task based on the ImageNet22k dataset. For SSLD, we used the `Top-k per class` data sampling scheme [3]. Specific steps are as follows. +     * Deduplication of training set. We first deduplicate the ImageNet22k dataset and the ImageNet1k validation set based on the SIFT feature similarity matching method to prevent the added ImageNet22k training set from containing the ImageNet1k validation set images. Finally we removed 4511 similar images. Similar pictures with partial filtering are shown below. + + ![](../../../images/distillation/22k_1k_val_compare_w_sift.png) + + * Obtain the soft label of the ImageNet22k dataset. For the ImageNet22k dataset after deduplication, we use the `ResNeXt101_32x16d_wsl` model to make predictions to obtain the soft label of each image. +     * Top-k data selection. There contains 1000 categories in ImageNet1k dataset. For each category, we find out images in the category with Top-k highest score, and finally generate a dataset whose image number does not exceed `1000 * k` (For some categories, there may contain less than k images). +     * The selected images are merged with the ImageNet1k training set to form the new dataset used for the final distillation model training, which contains 5 million images in all. + +## Experiments + +The distillation solution that PaddleClas provides is combining common training with finetuning. Given a suitable teacher model, the large dataset(5 million) is used for common training and the ImageNet1k dataset is used for finetuning. + +### Choice of teacher model + +In order to verify the influence of the model size difference between the teacher model and the student model on the distillation results as well as the teacher model accuracy, we conducted several experiments. The training strategy is unified as follows: `cosine_decay_warmup, lr = 1.3, epoch = 120, bs = 2048`, and the student models are all trained from scratch. + + +|Teacher Model | Teacher Top1 | Student Model | Student Top1| +|- |:-: |:-: | :-: | +| ResNeXt101_32x16d_wsl | 84.2% | MobileNetV3_large_x1_0 | 75.78% | +| ResNet50_vd | 79.12% | MobileNetV3_large_x1_0 | 75.60% | +| ResNet50_vd | 82.35% | MobileNetV3_large_x1_0 | 76.00% | + + +It can be shown from the table that: + +> When the teacher model structure is the same, the higher the teacher model accuracy, the better the final student model will be. +> +> The size difference between the teacher model and the student model should not be too large, otherwise it will decrease the accuracy of the distillation results. + +Therefore, during distillation, for the ResNet series student model, we use `ResNeXt101_32x16d_wsl` as the teacher model; for the MobileNet series student model, we use` ResNet50_vd_SSLD` as the teacher model. + + +### Distillation using large-scale dataset + +Training process is carried out on the large-scale dataset with 5 million images. Specifically, the following table shows more details of different models. + +|Student Model | num_epoch | l2_ecay | batch size/gpu cards | base lr | learning rate decay | top1 acc | +| - |:-: |:-: | :-: |:-: |:-: |:-: | +| MobileNetV1 | 360 | 3e-5 | 4096/8 | 1.6 | cosine_decay_warmup | 77.65% | +| MobileNetV2 | 360 | 1e-5 | 3072/8 | 0.54 | cosine_decay_warmup | 76.34% | +| MobileNetV3_large_x1_0 | 360 | 1e-5 | 5760/24 | 3.65625 | cosine_decay_warmup | 78.54% | +| MobileNetV3_small_x1_0 | 360 | 1e-5 | 5760/24 | 3.65625 | cosine_decay_warmup | 70.11% | +| ResNet50_vd | 360 | 7e-5 | 1024/32 | 0.4 | cosine_decay_warmup | 82.07% | +| ResNet101_vd | 360 | 7e-5 | 1024/32 | 0.4 | cosine_decay_warmup | 83.41% | +| Res2Net200_vd_26w_4s | 360 | 4e-5 | 1024/32 | 0.4 | cosine_decay_warmup | 84.82% | + +### finetuning using ImageNet1k + +Finetuning is carried out on ImageNet1k dataset to restore distribution between training set and test set. the following table shows more details of finetuning. + + +|Student Model | num_epoch | l2_ecay | batch size/gpu cards | base lr | learning rate decay | top1 acc | +| - |:-: |:-: | :-: |:-: |:-: |:-: | +| MobileNetV1 | 30 | 3e-5 | 4096/8 | 0.016 | cosine_decay_warmup | 77.89% | +| MobileNetV2 | 30 | 1e-5 | 3072/8 | 0.0054 | cosine_decay_warmup | 76.73% | +| MobileNetV3_large_x1_0 | 30 | 1e-5 | 2048/8 | 0.008 | cosine_decay_warmup | 78.96% | +| MobileNetV3_small_x1_0 | 30 | 1e-5 | 6400/32 | 0.025 | cosine_decay_warmup | 71.28% | +| ResNet50_vd | 60 | 7e-5 | 1024/32 | 0.004 | cosine_decay_warmup | 82.39% | +| ResNet101_vd | 30 | 7e-5 | 1024/32 | 0.004 | cosine_decay_warmup | 83.73% | +| Res2Net200_vd_26w_4s | 360 | 4e-5 | 1024/32 | 0.004 | cosine_decay_warmup | 85.13% | + +### Data agmentation and Fix strategy + +* Based on experiments mentioned above, we add AutoAugment [4] during training process, and reduced l2_decay from 4e-5 t 2e-5. Finally, the Top-1 accuracy on ImageNet1k dataset can reach 82.99%, with 0.6% improvement compared to the standard SSLD distillation strategy. + +* For image classsification tasks, The model accuracy can be further improved when the test scale is 1.15 times that of training[5]. For the 82.99% ResNet50_vd pretrained model, it comes to 83.7% using 320x320 for the evaluation. We use Fix strategy to finetune the model with the training scale set as 320x320. During the process, the pre-preocessing pipeline is same for both training and test. All the weights except the fully connected layer are freezed. Finally the top-1 accuracy comes to **84.0%**. + +### Some phenomena during the experiment + +In the prediction process, the average value and variance of the batch norm are obtained by loading the pretrained model (set its mode as test mode). In the training process, batch norm is obtained by counting the information of the current batch (set its mode as train mode) and calculating the moving average with the historical saved information. In the distillation task, we found that through the train mode, In the distillation task, we found that the real-time change of the bn parameter of the teacher model to guide the student model is better than the student model obtained through the test mode distillation. The following is a set of experimental results. Therefore, in this distillation scheme, we use train mode to get the soft label of the teacher model. + +|Teacher Model | Teacher Top1 | Student Model | Student Top1| +|- |:-: |:-: | :-: | +| ResNet50_vd | 82.35% | MobileNetV3_large_x1_0 | 76.00% | +| ResNet50_vd | 82.35% | MobileNetV3_large_x1_0 | 75.84% | + +## Application of the distillation model + +### Instructions + +* Adjust the learning rate of the middle layer. The middle layer feature map of the model obtained by distillation is more refined. Therefore, when the distillation model is used as the pretrained model in other tasks, if the same learning rate as before is adopted, it is easy to destroy the features. If the learning rate of the overall model training is reduced, it will bring about the problem of slow convergence. Therefore, we use the strategy of adjusting the learning rate of the middle layer. specifically: +    * For ResNet50_vd, we set up a learning rate list. The three conv2d convolution parameters before the resiual block have a uniform learning rate multiple, and the four resiual block conv2d have theirs own learning rate parameters, respectively. 5 values need to be set in the list. By the experiment, we find that when used for transfer learning finetune classification model, the learning rate list with `[0.1,0.1,0.2,0.2,0.3]` performs better in most tasks; while in the object detection tasks, `[0.05, 0.05, 0.05, 0.1, 0.15]` can bring greater accuracy gains. +    * For MoblileNetV3_large_x1_0, because it contains 15 blocks, we set each 3 blocks to share a learning rate, so 5 learning rate values are required. We find that in classification and detection tasks, the learning rate list with `[0.25, 0.25, 0.5, 0.5, 0.75]` performs better in most tasks. +* Appropriate l2 decay. Different l2 decay values are set for different models during training. In order to prevent overfitting, l2 decay is ofen set as large for large models. L2 decay is set as `1e-4` for ResNet50, and `1e-5 ~ 4e-5` for MobileNet series models. L2 decay needs also to be adjusted when applied in other tasks. Taking Faster_RCNN_MobiletNetV3_FPN as an example, we found that only modifying l2 decay can bring up to 0.5% accuracy (mAP) improvement on the COCO2017 dataset. + + +### Transfer learning + +* To verify the effect of the SSLD pretrained model in transfer learning, we carried out experiments on 10 small datasets. Here, in order to ensure the comparability of the experiment, we use the standard preprocessing process trained by the ImageNet1k dataset. For the distillation model, we also add a simple search method for the learning rate of the middle layers of the distillation pretrained model. +* For ResNet50_vd, the baseline pretrained model Top-1 Acc is 79.12%, the other parameters are got by grid search. For distillation pretrained model, we add learning rate of the middle layers into the search space. The following table shows the results. + +| Dataset | Model | Baseline Top1 Acc | Distillation Model Finetune | +|- |:-: |:-: | :-: | +| Oxford102 flowers | ResNete50_vd | 97.18% | 97.41% | +| caltech-101 | ResNete50_vd | 92.57% | 93.21% | +| Oxford-IIIT-Pets | ResNete50_vd | 94.30% | 94.76% | +| DTD | ResNete50_vd | 76.48% | 77.71% | +| fgvc-aircraft-2013b | ResNete50_vd | 88.98% | 90.00% | +| Stanford-Cars | ResNete50_vd | 92.65% | 92.76% | +| SUN397 | ResNete50_vd | 64.02% | 68.36% | +| cifar100 | ResNete50_vd | 86.50% | 87.58% | +| cifar10 | ResNete50_vd | 97.72% | 97.94% | +| Food-101 | ResNete50_vd | 89.58% | 89.99% | + +* It can be seen that on the above 10 datasets, combined with the appropriate middle layer learning rate, the distillation pretrained model can bring an average accuracy improvement of more than 1%. + +### Object detection + + +Based on the two-stage Faster/Cascade RCNN model, we verify the effect of the pretrained model obtained by distillation. + +* ResNet50_vd + +Training scale and test scale are set as 640x640, and some of the ablationstudies are as follows. + + +| Model | train/test scale | pretrain top1 acc | feature map lr | coco mAP | +|- |:-: |:-: | :-: | :-: | +| Faster RCNN R50_vd FPN | 640/640 | 79.12% | [1.0,1.0,1.0,1.0,1.0] | 34.8% | +| Faster RCNN R50_vd FPN | 640/640 | 79.12% | [0.05,0.05,0.1,0.1,0.15] | 34.3% | +| Faster RCNN R50_vd FPN | 640/640 | 82.18% | [0.05,0.05,0.1,0.1,0.15] | 36.3% | + + +It can be seen here that for the baseline pretrained model, excessive adjustment of the middle-layer learning rate actually reduces the performance of the detection model. Based on this distillation model, we also provide a practical server-side detection solution. The detailed configuration and training code are open source, more details can be refer to [PaddleDetection] (https://github.com/PaddlePaddle/PaddleDetection/tree/master/configs/rcnn_enhance). + + +## Practice + +This section will introduce the SSLD distillation experiments in detail based on the ImageNet-1K dataset. If you want to experience this method quickly, you can refer to [** Quick start PaddleClas in 30 minutes**] (../../tutorials/quick_start_en.md), whose dataset is set as Flowers102. + + + +### Configuration + + + +#### Distill MobileNetV3_small_x1_0 using MobileNetV3_large_x1_0 + +An example of SSLD distillation is provided here. The configuration file of `MobileNetV3_large_x1_0` distilling `MobileNetV3_small_x1_0` is provided in `ppcls/configs/ImageNet/Distillation/mv3_large_x1_0_distill_mv3_small_x1_0.yaml`, and the user can directly replace the path of the configuration file in `tools/train.sh` to use it. + +Configuration of distilling `MobileNetV3_large_x1_0` using `MobileNetV3_small_x1_0` is as follows. + +```yaml +Arch: + name: "DistillationModel" + # if not null, its lengths should be same as models + pretrained_list: + # if not null, its lengths should be same as models + freeze_params_list: + - True + - False + models: + - Teacher: + name: MobileNetV3_large_x1_0 + pretrained: True + use_ssld: True + - Student: + name: MobileNetV3_small_x1_0 + pretrained: False + + infer_model_name: "Student" +``` + +In configuration file, the `freeze_params_list` needs to specify whether the model needs to freeze the parameters, the `models` needs to specify the teacher model and the student model, and the teacher model needs to load the pretrained model. The user can directly change the model here. + +### Begin to train the network + +If everything is ready, users can begin to train the network using the following command. + +```bash + +python -m paddle.distributed.launch \ + --selected_gpus="0,1,2,3" \ + --log_dir=mv3_large_x1_0_distill_mv3_small_x1_0 \ + tools/train.py \ + -c ./ppcls/configs/ImageNet/Distillation/mv3_large_x1_0_distill_mv3_small_x1_0.yaml +``` + +### Note + +* Before using SSLD, users need to train a teacher model on the target dataset firstly. The teacher model is used to guide the training of the student model. + +* If the student model is not loaded with a pretrained model, the other hyperparameters of the training can refer to the hyperparameters trained by the student model on ImageNet-1k. If the student model is loaded with the pre-trained model, the learning rate can be adjusted to `1/100~1/10` of the standard learning rate. + +* In the process of SSLD distillation, the student model only learns the soft label, which makes the training process more difficult. It is recommended that the value of `l2_decay` can be decreased appropriately to obtain higher accuracy of the validation set. + +* If users are going to add unlabeled training data, just the training list textfile needs to be adjusted for more data. + + + +> If this document is helpful to you, welcome to star our project: [https://github.com/PaddlePaddle/PaddleClas](https://github.com/PaddlePaddle/PaddleClas) + + +## Reference + +[1] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015. + +[2] Bagherinezhad H, Horton M, Rastegari M, et al. Label refinery: Improving imagenet classification through label progression[J]. arXiv preprint arXiv:1805.02641, 2018. + +[3] Yalniz I Z, Jégou H, Chen K, et al. Billion-scale semi-supervised learning for image classification[J]. arXiv preprint arXiv:1905.00546, 2019. + +[4] Cubuk E D, Zoph B, Mane D, et al. Autoaugment: Learning augmentation strategies from data[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2019: 113-123. + +[5] Touvron H, Vedaldi A, Douze M, et al. Fixing the train-test resolution discrepancy[C]//Advances in Neural Information Processing Systems. 2019: 8250-8260. diff --git a/src/PaddleClas/docs/en/advanced_tutorials/distillation/index.rst b/src/PaddleClas/docs/en/advanced_tutorials/distillation/index.rst new file mode 100644 index 0000000..5548512 --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/distillation/index.rst @@ -0,0 +1,7 @@ +distillation +================================ + +.. toctree:: + :maxdepth: 3 + + distillation_en.md diff --git a/src/PaddleClas/docs/en/advanced_tutorials/how_to_contribute_en.md b/src/PaddleClas/docs/en/advanced_tutorials/how_to_contribute_en.md new file mode 100644 index 0000000..4502bda --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/how_to_contribute_en.md @@ -0,0 +1,304 @@ +# How to Contribute to the PaddleClas Community + +------ + +## Catalogue + +- [1. How to Contribute Code](#1) + - [1.1 Branches of PaddleClas](#1.1) + - [1.2 Commit Code to PaddleClas](#1.2) + - [1.2.1 Codes of Fork and Clone](#1.2.1) + - [1.2.2 Connect to the Remote Repository](#1.2.2) + - [1.2.3 Create the Local Branch](#1.2.3) + - [1.2.4 Employ Pre-commit Hook](#1.2.4) + - [1.2.5 Modify and Commit Code](#1.2.5) + - [1.2.6 Keep the Local Repository Updated](#1.2.6) + - [1.2.7 Push to Remote Repository](#1.2.7) + - [1.2.8 Commit Pull Request](#1.2.8) + - [1.2.9 CLA and Unit Test](#1.2.9) + - [1.2.10 Delete Branch](#1.2.10) + - [1.2.11 Conventions](#1.2.11) +- [2. Summary](#2) +- [3. Inferences](#3) + + + +## 1. How to Contribute Code + + + +### 1.1 Branches of PaddleClas + +PaddleClas maintains the following two branches: + +- release/x.x series: Stable release branches, which are tagged with the release version of Paddle in due course. +The latest and the default branch is the release/2.3, which is compatible with Paddle v2.1.0. +The branch of release/x.x series will continue to grow with future iteration, +and the latest release will be maintained by default, while the former one will fix bugs with no other branches covered. +- develop : developing branch, which is adapted to the develop version of Paddle and is mainly used for +developing new functions. A good choice for secondary development. +To ensure that the develop branch can pull out the release/x.x when needed, +only the API that is valid in Paddle's latest release branch can be adopted for its code. +In other words, if a new API has been developed in this branch but not yet in the release, +please do not use it in PaddleClas. Apart from that, features that do not involve the performance optimizations, +parameter adjustments, and policy updates of the API can be developed normally. + +The historical branches of PaddleClas will not be maintained, but will be remained for the existing users. + +- release/static: This branch was used for static graph development and testing, +and is currently compatible with >=1.7 versions of Paddle. +It is still practicable for the special need of adapting an old version of Paddle, +but the code will not be updated except for bug fixing. +- dygraph-dev: This branch will no longer be maintained and accept no new code. +Please transfer to the develop branch as soon as possible. + +PaddleClas welcomes code contributions to the repo, and the basic process is detailed in the next part. + + + +### 1.2 Commit the Code to PaddleClas + + + +#### 1.2.1 Codes of Fork and Clone + +- Skip to the home page of [PaddleClas GitHub](https://github.com/PaddlePaddle/PaddleClas) and click the +Fork button to generate a repository in your own directory, such as `https://github.com/USERNAME/PaddleClas`. + +[](../../images/quick_start/community/001_fork.png) + +- Clone the remote repository to local + +```shell +# Pull the code of the develop branch +git clone https://github.com/USERNAME/PaddleClas.git -b develop +cd PaddleClas +``` + +Obtain the address below + +[](../../images/quick_start/community/002_clone.png) + + + +#### 1.2.2 Connect to the Remote Repository + +First check the current information of the remote repository with `git remote -v`. + +```shell +origin https://github.com/USERNAME/PaddleClas.git (fetch) +origin https://github.com/USERNAME/PaddleClas.git (push) +``` + +The above information only contains the cloned remote repository, +which is the PaddleClas under your username. Then we create a remote host of the original PaddleClas repository named upstream. + +```shell +git remote add upstream https://github.com/PaddlePaddle/PaddleClas.git +``` + +Adopt `git remote -v` to view the current information of the remote repository, +and 2 remote repositories including origin and upstream can be found, as shown below. + +```shell +origin https://github.com/USERNAME/PaddleClas.git (fetch) +origin https://github.com/USERNAME/PaddleClas.git (push) +upstream https://github.com/PaddlePaddle/PaddleClas.git (fetch) +upstream https://github.com/PaddlePaddle/PaddleClas.git (push) +``` + +This is mainly to keep the local repository updated when committing a pull request (PR). + + + +#### 1.2.3 Create the Local Branch + +Run the following command to create a new local branch based on the current one. + +```shell +git checkout -b new_branch +``` + +Or you can create new ones based on remote or upstream branches. + +```shell +# Create the new_branch based on the develope of origin (unser remote repository) +git checkout -b new_branch origin/develop +# Create the new_branch base on the develope of upstream +# If you need to create a new branch from upstream, +# please first employ git fetch upstream to fetch the upstream code +git checkout -b new_branch upstream/develop +``` + +The following output shows that it has switched to the new branch with : + +``` +Branch new_branch set up to track remote branch develop from upstream. +Switched to a new branch 'new_branch' +``` + + + +#### 1.2.4 Employ Pre-commit Hook + +Paddle developers adopt the pre-commit tool to manage Git pre-commit hooks. +It helps us format the source code (C++, Python) and automatically check basic issues before committing +e.g., one EOL per file, no large files added to Git, etc. + +The pre-commit test is part of the unit tests in Travis-CI, +and PRs that do not satisfy the hook cannot be committed to PaddleClas. +Please install it first and run it in the current directory: + +``` +pip install pre-commit +pre-commit install +``` + +- **Note** + +1. Paddle uses clang-format to format C/C++ source code, please make sure `clang-format` has a version of 3.8 or higher. +2. `yapf` installed by `pip install pre-commit` and `conda install -c conda-forge pre-commit` is slightly different, +and the former one is chosen by PaddleClas developers. + + + +#### 1.2.5 Modify and Commit Code + +You can check the changed files via `git status`. Follow the steps below to commit the `README.md` of PaddleClas after modification: + +``` +git add README.md +pre-commit +``` + +Repeat the above steps until the pre-commit format check does not report an error, as shown below. + +[](../../images/quick_start/community/003_precommit_pass.png) + +Run the following command to commit. + +``` +git commit -m "your commit info" +``` + + + +#### 1.2.6 Keep the Local Repository Updated + +Get the latest code for upstream and update the current branch. +The upstream here is from the `Connecting to a remote repository` part in section 1.2. + +``` +git fetch upstream +# If you want to commit to another branch, please pull the code from another branch of upstream, in this case it is develop +git pull upstream develop +``` + + + +#### 1.2.7 Push to Remote Repository + +``` +git push origin new_branch +``` + + + +#### 1.2.8 Commit Pull Request + +Click new pull request and select the local branch and the target branch, +as shown in the following figure. In the description of the PR, fill out what the PR accomplishes. +Next, wait for the review, and if any changes are required, +update the corresponding branch in origin by referring to the above steps. + +[](../../images/quick_start/community/004_create_pr.png) + + + +#### 1.2.9 CLA and Unit Test + +- When you first commit a Pull Request to PaddlePaddle, +you are required to sign a CLA (Contributor License Agreement) to ensure that your code can be merged, +please follow the step below to sign CLA: + +1. Please examine the Check section of your PR, find license/cla, +and click the detail on the right side to enter the CLA website +2. Click `Sign in with GitHub to agree` on the CLA website, +and you will be redirected back to your Pull Request page when you are done. + + + +#### 1.2.10 Delete Branch + +- Delete remote branch + +When the PR is merged into the main repository, you can delete the remote branch from the PR page. + +You can also delete the remote branch using `git push origin :branch name`, e.g. + +``` +git push origin :new_branch +``` + +- Delete local branch + +``` +# Switch to the develop branch, otherwise the current branch cannot be deleted +git checkout develop + +# Delete new_branch +git branch -D new_branch +``` + + + +#### 1.2.11 Conventions + +To help official maintainers focus on the code itself when reviewing it, +please adhere to the following conventions each time you commit code: + +1. Please pass the unit test in Travis-CI first. +Otherwise, the submitted code may have problems and usually receive no official review. +2. Before committing a Pull Request: +Note the number of commits. + +Reason: If only one file is modified but more than a dozen commits are committed with a few changes for each, +this may overwhelm the reviewer for they need to check each and every commit for specific changes, +including the case that the changes between commits overwrite each other. + +Recommendation: Minimize the number of commits each time, and add the last commit with `git commit --amend`. +For multiple commits that have been pushed to a remote repository, please refer to +[squash commits after push](https://stackoverflow.com/questions/5667884/how-to-squash-commits-in-git-after-they-have-been-pushed). + +Please pay attention to the name of each commit: +it should reflect the content of the current commit without being too casual. + +3. If an issue is resolved, please add `fix #issue_number` to the first comment box of the Pull Request, +so that the corresponding issue will be closed automatically when the Pull Request is merged. Please choose the appropriate term with keywords such as close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved, please choose the appropriate term. See details in [Closing issues via commit messages](https://help.github.com/articles/closing-issues-via-commit-messages). + +In addition, please stick to the following convention to respond to reviewers' comments: + +1. Every review comment from the official maintainer is expected to be answered, +which will better enhance the contribution of the open source community. + +- If you agree with the review and finish the corresponding modification, please simply return Done; +- If you disagree with the review, please give your reasons. + +2. If there are plenty of review comments, + +- Please present the revision in general. +- Please reply with `start a review` instead of a direct approach, for it may be overwhelming to receive the email of every reply. + + + +## 2. Summary + +- The open source community relies on the contributions and feedback of developers and users. +We highly appreciate that and look forward to your valuable comments and Pull Requests to PaddleClas in the hope that together we can build a leading practical and comprehensive code repository for image recognition! + + + +## 3. References + +1. [Guide to PaddlePaddle Local Development](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/08_contribution/index_en.html) +2. [Committing PR to Open Source Framework](https://blog.csdn.net/vim_wj/article/details/78300239) diff --git a/src/PaddleClas/docs/en/advanced_tutorials/index.rst b/src/PaddleClas/docs/en/advanced_tutorials/index.rst new file mode 100644 index 0000000..e741733 --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/index.rst @@ -0,0 +1,12 @@ +advanced_tutorials +================================ + +.. toctree:: + :maxdepth: 2 + + DataAugmentation_en.md + distillation/index + multilabel/index + model_prune_quantization_en.md + code_overview_en.md + how_to_contribute_en.md diff --git a/src/PaddleClas/docs/en/advanced_tutorials/model_prune_quantization_en.md b/src/PaddleClas/docs/en/advanced_tutorials/model_prune_quantization_en.md new file mode 100644 index 0000000..1946cfa --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/model_prune_quantization_en.md @@ -0,0 +1,180 @@ +# Model Quantization and Pruning + +Complex models are conducive to better model performance, but they may also lead to certain redundancy. This section presents ways to streamline the model, including model quantization (quantization training and offline quantization) and model pruning. + +Model quantization reduces the full precision to a fixed number of points to lower the redundancy and achieve the purpose of simplifying the model computation and improving model inference performance. Model quantization can reduce the size of model parameters by converting its precision from FP32 to Int8 without losing model precision, followed by accelerated computation, creating a quantized model with more speed advantages when deployed on mobile devices. + +Model pruning decreases the number of model parameters by cutting out the unimportant convolutional kernels in the CNN, thus bringing down the computational complexity. + +This tutorial explains how to use PaddleSlim, PaddlePaddle's model compression library, for PaddleClas compression, i.e., pruning and quantization. [PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim) integrates a variety of common and leading model compression functions such as model pruning, quantization (including quantization training and offline quantization), distillation, and neural network search. If you are interested, please follow us and learn more. + +To start with, you are recommended to learn [PaddleClas Training](../models_training/classification_en.md) and [PaddleSlim](https://paddleslim.readthedocs.io/zh_CN/latest/index.html), see [Model Pruning and Quantization Algorithms](../algorithm_introduction/model_prune_quantization_en.md) for related pruning and quantization methods. + +------ + +## Catalogue + +- [1. Prepare the Environment](#1) + - [1.1 Install PaddleSlim](#1.1) + - [1.2 Prepare the Trained Model](#1.2) +- [2. Quick Start](#2) + - [2.1 Model Quantization](#2.1) + - [2.1.1 Online Quantization Training](#2.1.1) + - [2.1.2 Offline Quantization](#2.1.2) + - [2.2 Model Pruning](#2.2) +- [3. Export the Model](#3) +- [4. Deploy the Model](#4) +- [5. Hyperparameter Training](#5) + + + +## 1. Prepare the Environment + +Once a model has been trained, you can adopt quantization or pruning to further compress the model size and speed up the inference. + +Five steps are included： + +1. Install PaddleSlim +2. Prepare the trained the model +3. Compress the model +4. Export quantized inference model +5. Inference and deployment of the quantized model + + + +### 1.1 Install PaddleSlim + +- You can adopt pip install for installation. + +``` +pip install paddleslim -i https://pypi.tuna.tsinghua.edu.cn/simple +``` + +- You can also install it from the source code with the latest features of PaddleSlim. + +``` +git clone https://github.com/PaddlePaddle/PaddleSlim.git +cd Paddleslim +python3.7 setup.py install +``` + + + +### 1.2 Prepare the Trained Model + +PaddleClas offers a list of trained [models](../models/models_intro_en.md). If the model to be quantized is not in the list, you need to follow the [regular training](../models_training/classification_en.md) method to get the trained model. + + + +## 2. Quick Start + +Go to PaddleClas root directory + +```shell +cd PaddleClas +``` + +Related code for `slim` training has been integrated under `ppcls/engine/`, and the offline quantization code can be found in `deploy/slim/quant_post_static.py`. + + + +### 2.1 Model Quantization + +Quantization training includes offline and online training. Online quantitative training, the more effective one, requires loading a pre-trained model, which can be quantized after defining the strategy. + + + +#### 2.1.1 Online Quantization Training + +Try the following command： + +- CPU/Single GPU + +Take CPU for example, if you use GPU, change the `cpu` to `gpu`. + +``` +python3.7 tools/train.py -c ppcls/configs/slim/ResNet50_vd_quantization.yaml -o Global.device=cpu +``` + +The parsing of the `yaml` file is described in [reference document](../models_training/config_description_en.md). For accuracy, the `pretrained model` has already been adopted by the `yaml` file. + +- Launch in single-machine multi-card/ multi-machine multi-card mode + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3.7 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ppcls/configs/slim/ResNet50_vd_quantization.yaml +``` + + + +#### 2.1.2 Offline Quantization + +**Note**: Currently, the `inference model` exported from the trained model is a must for offline quantization. See the [tutorial](../inference_deployment/export_model_en.md) for general export of the `inference model`. + +Normally, offline quantization may lose more accuracy. + +After generating the `inference model`, the offline quantization is run as follows: + +```shell +python3.7 deploy/slim/quant_post_static.py -c ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml -o Global.save_inference_dir=./deploy/models/class_ResNet50_vd_ImageNet_infer +``` + +The `inference model` is stored in`Global.save_inference_dir`. + +Successfully executed, the `quant_post_static_model` folder is created in the `Global.save_inference_dir`, where the generated offline quantization models are stored and can be deployed directly without re-exporting the models. + + + +### 2.2 Model Pruning + +Trying the following command： + +- CPU/Single GPU + +Take CPU for example, if you use GPU, change the `cpu` to `gpu`. + +```shell +python3.7 tools/train.py -c ppcls/configs/slim/ResNet50_vd_prune.yaml -o Global.device=cpu +``` + +- Launch in single-machine single-card/ single-machine multi-card/ multi machine multi-card mode + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3.7 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ppcls/configs/slim/ResNet50_vd_prune.yaml +``` + + + +## 3. Export the Model + +Having obtained the saved model after online quantization training and pruning, it can be exported as an inference model for inference deployment. Here we take model pruning as an example: + +``` +python3.7 tools/export.py \ + -c ppcls/configs/slim/ResNet50_vd_prune.yaml \ + -o Global.pretrained_model=./output/ResNet50_vd/best_model \ + -o Global.save_inference_dir=./inference +``` + + + +## 4. Deploy the Model + +The exported model can be deployed directly using inference, please refer to [inference deployment](../inference_deployment/). + +You can also use PaddleLite's opt tool to convert the inference model to a mobile model for its mobile deployment. Please refer to [Mobile Model Deployment](../inference_deployment/paddle_lite_deploy_en.md) for more details. + + + +## 5. Hyperparameter Training + +- For quantization and pruning training, it is recommended to load the pre-trained model obtained from conventional training to accelerate the convergence of quantization training. +- For quantization training, it is recommended to modify the initial learning rate to `1/20~1/10` of the conventional training and the number of training epochs to `1/5~1/2`, while adding Warmup to the learning rate strategy. Please make no other modifications to the configuration information. +- For pruning training, the hyperparameter configuration is recommended to remain the same as the regular training. diff --git a/src/PaddleClas/docs/en/advanced_tutorials/multilabel/index.rst b/src/PaddleClas/docs/en/advanced_tutorials/multilabel/index.rst new file mode 100644 index 0000000..07e0a54 --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/multilabel/index.rst @@ -0,0 +1,7 @@ +Multilabel Classification +================================ + +.. toctree:: + :maxdepth: 3 + + multilabel_en.md \ No newline at end of file diff --git a/src/PaddleClas/docs/en/advanced_tutorials/multilabel/multilabel_en.md b/src/PaddleClas/docs/en/advanced_tutorials/multilabel/multilabel_en.md new file mode 100644 index 0000000..46a89f0 --- /dev/null +++ b/src/PaddleClas/docs/en/advanced_tutorials/multilabel/multilabel_en.md @@ -0,0 +1,92 @@ +# Multilabel classification quick start + +Based on the [NUS-WIDE-SCENE](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html) dataset which is a subset of NUS-WIDE dataset, you can experience multilabel of PaddleClas, include training, evaluation and prediction. Please refer to [Installation](../../installation/) to install at first. + +## Preparation + +* Enter PaddleClas directory + +``` +cd path_to_PaddleClas +``` + +* Create and enter `dataset/NUS-WIDE-SCENE` directory, download and decompress NUS-WIDE-SCENE dataset + +```shell +mkdir dataset/NUS-WIDE-SCENE +cd dataset/NUS-WIDE-SCENE +wget https://paddle-imagenet-models-name.bj.bcebos.com/data/NUS-SCENE-dataset.tar +tar -xf NUS-SCENE-dataset.tar +``` + +* Return `PaddleClas` root home + +``` +cd ../../ +``` + +## Training + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml +``` + +After training for 10 epochs, the best accuracy over the validation set should be around 0.95. + +## Evaluation + +```bash +python tools/eval.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + +## Prediction + +```bash +python3 tools/infer.py + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + +You will get multiple output such as the following: +``` +[{'class_ids': [6, 13, 17, 23, 26, 30], 'scores': [0.95683, 0.5567, 0.55211, 0.99088, 0.5943, 0.78767], 'file_name': './deploy/images/0517_2715693311.jpg', 'label_names': []}] +``` + +## Prediction based on prediction engine + +### Export model + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + +The default path of the inference model is under the current path `./inference` + +### Prediction based on prediction engine + +Enter the deploy directory: + +```bash +cd ./deploy +``` + +Prediction based on prediction engine: + +``` +python3 python/predict_cls.py \ + -c configs/inference_multilabel_cls.yaml +``` + +You will get multiple output such as the following: + +``` +0517_2715693311.jpg: class id(s): [6, 13, 17, 23, 26, 30], score(s): [0.96, 0.56, 0.55, 0.99, 0.59, 0.79], label_name(s): [] +``` diff --git a/src/PaddleClas/docs/en/algorithm_introduction/DataAugmentation_en.md b/src/PaddleClas/docs/en/algorithm_introduction/DataAugmentation_en.md new file mode 100644 index 0000000..9ada02f --- /dev/null +++ b/src/PaddleClas/docs/en/algorithm_introduction/DataAugmentation_en.md @@ -0,0 +1,264 @@ +# Data Augmentation +------ + +## Catalogue + + - [1. Introduction to data augmentation](#1) + - [2. Common data augmentation methods](#2) + - [2.1 Image Transformation](#2.1) + - [2.1.1 AutoAugment](#2.1.1) + - [2.1.2 RandAugment](#2.1.2) + - [2.1.3 TimmAutoAugment](#2.1.3) + - [2.2 Image Cropping](#2.2) + - [2.2.1 Cutout](#2.2.1) + - [2.2.2 RandomErasing](#2.2.2) + - [2.2.3 HideAndSeek](#2.2.3) + - [2.2.4 GridMask](#2.2.4) + - [2.3 Image mix](#2.3) + - [2.3.1 Mixup](#2.3.1) + - [2.3.2 Cutmix](#2.3.2) + + +## 1. Introduction to data augmentation + +Data augmentation is a commonly used regularization method in image classification task, which is often used in scenarios with insufficient data or large model. In this chapter, we mainly introduce 8 image augmentation methods besides standard augmentation methods. Users can apply these methods in their own tasks for better model performance. Under the same conditions, these augmentation methods' performance on ImageNet1k dataset is shown as follows. + +![](../../images/image_aug/main_image_aug.png) + + + +## 2. Common data augmentation methods + +If without special explanation, all the examples and experiments in this chapter are based on ImageNet1k dataset with the network input image size set as 224. + +The standard data augmentation pipeline in ImageNet classification tasks contains the following steps. + +1. Decode image, abbreviated as `ImageDecode`. +2. Randomly crop the image to size with 224x224, abbreviated as `RandCrop`. +3. Randomly flip the image horizontally, abbreviated as `RandFlip`. +4. Normalize the image pixel values, abbreviated as `Normalize`. +5. Transpose the image from `[224, 224, 3]`(HWC) to `[3, 224, 224]`(CHW), abbreviated as `Transpose`. +6. Group the image data(`[3, 224, 224]`) into a batch(`[N, 3, 224, 224]`), where `N` is the batch size. It is abbreviated as `Batch`. + + +Compared with the above standard image augmentation methods, the researchers have also proposed many improved image augmentation strategies. These strategies are to insert certain operations at different stages of the standard augmentation method, based on the different stages of operation. We divide it into the following three categories. + +1. Transformation. Perform some transformations on the image after `RandCrop`, such as AutoAugment and RandAugment. +2. Cropping. Perform some transformations on the image after `Transpose`, such as CutOut, RandErasing, HideAndSeek and GridMask. +3. Aliasing. Perform some transformations on the image after `Batch`, such as Mixup and Cutmix. + + +Visualization results of some images after augmentation are shown as follows. + +![](../../images/image_aug/image_aug_samples_s_en.jpg) + + +The following table shows more detailed information of the transformations. + + +| Method | Input | Output | Auto-
Augment\[1\] | Rand-
Augment\[2\] | CutOut\[3\] | Rand
Erasing\[4\] | HideAnd-
Seek\[5\] | GridMask\[6\] | Mixup\[7\] | Cutmix\[8\] | +|-------------|---------------------------|---------------------------|------------------|------------------|-------------|------------------|------------------|---------------|------------|------------| +| Image
Decode | Binary | (224, 224, 3)
uint8 | Y | Y | Y | Y | Y | Y | Y | Y | +| RandCrop | (:, :, 3)
uint8 | (224, 224, 3)
uint8 | Y | Y | Y | Y | Y | Y | Y | Y | +| **Process** | (224, 224, 3)
uint8 | (224, 224, 3)
uint8 | Y | Y | \- | \- | \- | \- | \- | \- | +| RandFlip | (224, 224, 3)
uint8 | (224, 224, 3)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| Normalize | (224, 224, 3)
uint8 | (3, 224, 224)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| Transpose | (224, 224, 3)
float32 | (3, 224, 224)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| **Process** | (3, 224, 224)
float32 | (3, 224, 224)
float32 | \- | \- | Y | Y | Y | Y | \- | \- | +| Batch | (3, 224, 224)
float32 | (N, 3, 224, 224)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| **Process** | (N, 3, 224, 224)
float32 | (N, 3, 224, 224)
float32 | \- | \- | \- | \- | \- | \- | Y | Y | + + + +PaddleClas integrates all the above data augmentation strategies. More details including principles and usage of the strategies are introduced in the following chapters. For better visualization, we use the following figure to show the changes after the transformations. And `RandCrop` is replaced with` Resize` for simplification. + +![](../../images/image_aug/test_baseline.jpeg) + + +### 2.1 Image Transformation + +Transformation means performing some transformations on the image after `RandCrop`. It mainly contains AutoAugment and RandAugment. + + +#### 2.1.1 AutoAugment + +Address：[https://arxiv.org/abs/1805.09501v1](https://arxiv.org/abs/1805.09501v1) + +Github repo：[https://github.com/DeepVoltaire/AutoAugment](https://github.com/DeepVoltaire/AutoAugment) + + +Unlike conventional artificially designed image augmentation methods, AutoAugment is an image augmentation solution suitable for a specific data set found by certain search algorithm in the search space of a series of image augmentation sub-strategies. For the ImageNet dataset, the final data augmentation solution contains 25 sub-strategy combinations. Each sub-strategy contains two transformations. For each image, a sub-strategy combination is randomly selected and then determined with a certain probability Perform each transformation in the sub-strategy. + +The images after `AutoAugment` are as follows. + +![](../../images/image_aug/test_autoaugment.jpeg) + + +#### 2.1.2 RandAugment + +Address: [https://arxiv.org/pdf/1909.13719.pdf](https://arxiv.org/pdf/1909.13719.pdf) + +Github repo: [https://github.com/heartInsert/randaugment](https://github.com/heartInsert/randaugment) + + +The search method of `AutoAugment` is relatively violent. Searching for the optimal strategy for this data set directly on the data set requires a lot of computation. In `RandAugment`, the author found that on the one hand, for larger models and larger datasets, the gains generated by the augmentation method searched using `AutoAugment` are smaller. On the other hand, the searched strategy is limited to certain dataset, which has poor generalization performance and not sutable for other datasets. + +In `RandAugment`, the author proposes a random augmentation method. Instead of using a specific probability to determine whether to use a certain sub-strategy, all sub-strategies are selected with the same probability. The experiments in the paper also show that this method performs well even for large models. + +The images after `RandAugment` are as follows. + +![](../../images/image_aug/test_randaugment.jpeg) + + +#### 2.1.3 TimmAutoAugment + +Github open source code address: [https://github.com/rwightman/pytorch-image-models/blob/master/timm/data/auto_augment.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/data/auto_augment.py) + +`TimmAutoAugment` is an improvement of AutoAugment and RandAugment by open source authors. Facts have proved that it has better performance on many visual tasks. At present, most VisionTransformer models are implemented based on TimmAutoAugment. + + +### 2.2 Image Cropping + +Cropping means performing some transformations on the image after `Transpose`, setting pixels of the cropped area as certain constant. It mainly contains CutOut, RandErasing, HideAndSeek and GridMask. + +Image cropping methods can be operated before or after normalization. The difference is that if we crop the image before normalization and fill the areas with 0, the cropped areas' pixel values will not be 0 after normalization, which will cause grayscale distribution change of the data. + +The above-mentioned cropping transformation ideas are the similar, all to solve the problem of poor generalization ability of the trained model on occlusion images, the difference lies in that their cropping details. + + +#### 2.2.1 Cutout + +Address: [https://arxiv.org/abs/1708.04552](https://arxiv.org/abs/1708.04552) + +Github repo: [https://github.com/uoguelph-mlrg/Cutout](https://github.com/uoguelph-mlrg/Cutout) + + +Cutout is a kind of dropout, but occludes input image rather than feature map. It is more robust to noise than noise. Cutout has two advantages: (1) Using Cutout, we can simulate the situation when the subject is partially occluded. (2) It can promote the model to make full use of more content in the image for classification, and prevent the network from focusing only on the saliency area, thereby causing overfitting. + +The images after `Cutout` are as follows. + +![](../../images/image_aug/test_cutout.jpeg) + + +#### 2.2.2 RandomErasing + +Address: [https://arxiv.org/pdf/1708.04896.pdf](https://arxiv.org/pdf/1708.04896.pdf) + +Github repo: [https://github.com/zhunzhong07/Random-Erasing](https://github.com/zhunzhong07/Random-Erasing) + +RandomErasing is similar to the Cutout. It is also to solve the problem of poor generalization ability of the trained model on images with occlusion. The author also pointed out in the paper that the way of random cropping is complementary to random horizontal flipping. The author also verified the effectiveness of the method on pedestrian re-identification (REID). Unlike `Cutout`, in` `, `RandomErasing` is operateed on the image with a certain probability, size and aspect ratio of the generated mask are also randomly generated according to pre-defined hyperparameters. + +The images after `RandomErasing` are as follows. + +![](../../images/image_aug/test_randomerassing.jpeg) + + +#### 2.2.3 HideAndSeek + +Address: [https://arxiv.org/pdf/1811.02545.pdf](https://arxiv.org/pdf/1811.02545.pdf) + +Github repo: [https://github.com/kkanshul/Hide-and-Seek](https://github.com/kkanshul/Hide-and-Seek) + + +Images are divided into some patches for `HideAndSeek` and masks are generated with certain probability for each patch. The meaning of the masks in different areas is shown in the figure below. + +![](../../images/image_aug/hide-and-seek-visual.png) + +The images after `HideAndSeek` are as follows. + +![](../../images/image_aug/gridmask-0.png) + + +#### 2.2.4 GridMask +Address：[https://arxiv.org/abs/2001.04086](https://arxiv.org/abs/2001.04086) + +Github repo：[https://github.com/akuxcw/GridMask](https://github.com/akuxcw/GridMask) + + +The author points out that the previous method based on image cropping has two problems, as shown in the following figure: + +1. Excessive deletion of the area may cause most or all of the target subject to be deleted, or cause the context information loss, resulting in the images after enhancement becoming noisy data. +2. Reserving too much area has little effect on the object and context. + +![](../../images/image_aug/hide-and-seek-visual.png) + +Therefore, it is the core problem to be solved how to +if you avoid over-deletion or over-retention becomes the core problem to be solved. + +`GridMask` is to generate a mask with the same resolution as the original image and multiply it with the original image. The mask grid and size are adjusted by the hyperparameters. + +In the training process, there are two methods to use: +1. Set a probability p and use the GridMask to augment the image with probability p from the beginning of training. +2. Initially set the augmentation probability to 0, and the probability is increased with number of iterations from 0 to p. + +It shows that the second method is better. + +The images after `GridMask` are as follows. + +![](../../images/image_aug/test_gridmask.jpeg) + + +### 2.3 Image mix + +mix means performing some transformations on the image after `Batch`, which contains Mixup and Cutmix. + +Data augmentation methods introduced before are based on single image while mixing is carried on a certain batch to generate a new batch. + + +#### 2.3.1 Mixup + +Address: [https://arxiv.org/pdf/1710.09412.pdf](https://arxiv.org/pdf/1710.09412.pdf) + +Github repo: [https://github.com/facebookresearch/mixup-cifar10](https://github.com/facebookresearch/mixup-cifar10) + +Mixup is the first solution for image aliasing, it is easy to realize and performs well not only on image classification but also on object detection. Mixup is usually carried out in a batch for simplification, so as `Cutmix`. + +The images after `Mixup` are as follows. + +![](../../images/image_aug/test_mixup.png) + + +#### 2.3.2 Cutmix + +Address: [https://arxiv.org/pdf/1905.04899v2.pdf](https://arxiv.org/pdf/1905.04899v2.pdf) + +Github repo: [https://github.com/clovaai/CutMix-PyTorch](https://github.com/clovaai/CutMix-PyTorch) + +Unlike `Mixup` which directly adds two images, for Cutmix, an `ROI` is cut out from one image and +Cutmix randomly cuts out an `ROI` from one image, and then covered onto the corresponding area in the another image. + +The images after `Cutmix` are as follows. + +![](../../images/image_aug/test_cutmix.png) + +For the practical part of data augmentation, please refer to [Data Augmentation Practice](../advanced_tutorials/DataAugmentation_en.md). + +## Reference + +[1] Cubuk E D, Zoph B, Mane D, et al. Autoaugment: Learning augmentation strategies from data[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2019: 113-123. + +[2] Cubuk E D, Zoph B, Shlens J, et al. Randaugment: Practical automated data augmentation with a reduced search space[J]. arXiv preprint arXiv:1909.13719, 2019. + +[3] DeVries T, Taylor G W. Improved regularization of convolutional neural networks with cutout[J]. arXiv preprint arXiv:1708.04552, 2017. + +[4] Zhong Z, Zheng L, Kang G, et al. Random erasing data augmentation[J]. arXiv preprint arXiv:1708.04896, 2017. + +[5] Singh K K, Lee Y J. Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization[C]//2017 IEEE international conference on computer vision (ICCV). IEEE, 2017: 3544-3553. + +[6] Chen P. GridMask Data Augmentation[J]. arXiv preprint arXiv:2001.04086, 2020. + +[7] Zhang H, Cisse M, Dauphin Y N, et al. mixup: Beyond empirical risk minimization[J]. arXiv preprint arXiv:1710.09412, 2017. + +[8] Yun S, Han D, Oh S J, et al. Cutmix: Regularization strategy to train strong classifiers with localizable features[C]//Proceedings of the IEEE International Conference on Computer Vision. 2019: 6023-6032. + +[test_baseline]: ../../../images/image_aug/test_baseline.jpeg +[test_autoaugment]: ../../../images/image_aug/test_autoaugment.jpeg +[test_cutout]: ../../../images/image_aug/test_cutout.jpeg +[test_gridmask]: ../../../images/image_aug/test_gridmask.jpeg +[gridmask-0]: ../../../images/image_aug/gridmask-0.png +[test_hideandseek]: ../../../images/image_aug/test_hideandseek.jpeg +[test_randaugment]: ../../../images/image_aug/test_randaugment.jpeg +[test_randomerassing]: ../../../images/image_aug/test_randomerassing.jpeg +[hide_and_seek_mask_expanation]: ../../../images/image_aug/hide-and-seek-visual.png +[test_mixup]: ../../../images/image_aug/test_mixup.png +[test_cutmix]: ../../../images/image_aug/test_cutmix.png diff --git a/src/PaddleClas/docs/en/algorithm_introduction/ImageNet_models_en.md b/src/PaddleClas/docs/en/algorithm_introduction/ImageNet_models_en.md new file mode 100644 index 0000000..c9d0a72 --- /dev/null +++ b/src/PaddleClas/docs/en/algorithm_introduction/ImageNet_models_en.md @@ -0,0 +1,599 @@ +# ImageNet Model zoo overview + +## Catalogue + +- [1. Model library overview diagram](#1) +- [2. SSLD pretrained models](#2) + - [2.1 Server-side knowledge distillation model](#2.1) + - [2.2 Mobile-side knowledge distillation model](#2.2) + - [2.3 Intel-CPU-side knowledge distillation model](#2.3) +- [3. PP-LCNet series](#3) +- [4. ResNet series](#4) +- [5. Mobile series](#5) +- [6. SEResNeXt and Res2Net series](#6) +- [7. DPN and DenseNet series](#7) +- [8. HRNet series](#8) +- [9. Inception series](#9) +- [10. EfficientNet ans ResNeXt101_wsl series](#10) +- [11. ResNeSt and RegNet series](#11) +- [12. ViT and DeiT series](#12) +- [13. RepVGG series](#13) +- [14. MixNet series](#14) +- [15. ReXNet series](#15) +- [16. SwinTransformer series](#16) +- [17. LeViT series](#17) +- [18. Twins series](#18) +- [19. HarDNet series](#19) +- [20. DLA series](#20) +- [21. RedNet series](#21) +- [22. TNT series](#22) +- [23. Other models](#23) +- [Reference](#reference) + + + +## 1. Model library overview diagram + +Based on the ImageNet-1k classification dataset, the 37 classification network structures supported by PaddleClas and the corresponding 217 image classification pretrained models are shown below. Training trick, a brief introduction to each series of network structures, and performance evaluation will be shown in the corresponding chapters. The evaluation environment is as follows. + +* Arm CPU evaluation environment is based on Snapdragon 855 (SD855). +* Intel CPU evaluation environment is based on Intel(R) Xeon(R) Gold 6148. +* The GPU evaluation speed is measured by running 2100 times under the FP32+TensorRT configuration (excluding the warmup time of the first 100 times). +* FLOPs and Params are calculated by `paddle.flops()` (PaddlePaddle version is 2.2) + +Curves of accuracy to the inference time of common server-side models are shown as follows. + +![](../../images/models/V100_benchmark/v100.fp32.bs1.main_fps_top1_s.png) + +Curves of accuracy to the inference time of common mobile-side models are shown as follows. + +![](../../images/models/mobile_arm_top1.png) + +Curves of accuracy to the inference time of some VisionTransformer models are shown as follows. + +![](../../images/models/V100_benchmark/v100.fp32.bs1.visiontransformer.png) + + + +## 2. SSLD pretrained models +Accuracy and inference time of the prtrained models based on SSLD distillation are as follows. More detailed information can be refered to [SSLD distillation tutorial](../advanced_tutorials/distillation/distillation_en.md). + + + +### 2.1 Server-side knowledge distillation model + +| Model | Top-1 Acc | Reference
Top-1 Acc | Acc gain | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|---------------------|-----------|-----------|---------------|----------------|-----------|----------|-----------|-----------------------------------|-----------------------------------|-----------------------------------| +| ResNet34_vd_ssld | 0.797 | 0.760 | 0.037 | 2.00 | 3.28 | 5.84 | 3.93 | 21.84 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_vd_ssld_pretrained.pdparams)   | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_vd_ssld_infer.tar)   | +| ResNet50_vd_ssld | 0.830 | 0.792 | 0.039 | 2.60 | 4.86 | 7.63 | 4.35 | 25.63 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_ssld_infer.tar) | +| ResNet101_vd_ssld | 0.837 | 0.802 | 0.035 | 4.43 | 8.25 | 12.60 | 8.08 | 44.67 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_vd_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_vd_ssld_infer.tar) | +| Res2Net50_vd_26w_4s_ssld | 0.831 | 0.798 | 0.033 | 3.59 | 6.35 | 9.50 | 4.28 | 25.76 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_vd_26w_4s_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_vd_26w_4s_ssld_infer.tar) | +| Res2Net101_vd_
26w_4s_ssld | 0.839 | 0.806 | 0.033 | 6.34 | 11.02 | 16.13 | 8.35 | 45.35 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net101_vd_26w_4s_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net101_vd_26w_4s_ssld_infer.tar) | +| Res2Net200_vd_
26w_4s_ssld | 0.851 | 0.812 | 0.049 | 11.45 | 19.77 | 28.81 | 15.77 | 76.44 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net200_vd_26w_4s_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net200_vd_26w_4s_ssld_infer.tar) | +| HRNet_W18_C_ssld | 0.812 | 0.769 | 0.043 | 6.66 | 8.94 | 11.95 | 4.32 | 21.35 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W18_C_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W18_C_ssld_infer.tar) | +| HRNet_W48_C_ssld | 0.836 | 0.790 | 0.046 | 11.07 | 17.06 | 27.28 | 17.34 | 77.57 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W48_C_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W48_C_ssld_infer.tar) | +| SE_HRNet_W64_C_ssld | 0.848 | - | - | 17.11 | 26.87 | 43.24 | 29.00 | 129.12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/SE_HRNet_W64_C_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_HRNet_W64_C_ssld_infer.tar) | + + + +### 2.2 Mobile-side knowledge distillation model + +| Model | Top-1 Acc | Reference
Top-1 Acc | Acc gain | SD855 time(ms)
bs=1, thread=1 | SD855 time(ms)
bs=1, thread=2 | SD855 time(ms)
bs=1, thread=4 | FLOPs(M) | Params(M) | Model大小(M) | Pretrained Model Download Address | Inference Model Download Address | +|---------------------|-----------|-----------|---------------|----------------|-----------|----------|-----------|-----------------------------------|-----------------------------------|-----------------------------------|-----------------------------------| +| MobileNetV1_ssld | 0.779 | 0.710 | 0.069 | 30.24 | 17.86 | 10.30 | 578.88 | 4.25 | 16 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_ssld_infer.tar) | +| MobileNetV2_ssld | 0.767 | 0.722 | 0.045 | 20.74 | 12.71 | 8.10 | 327.84 | 3.54 | 14 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_ssld_infer.tar) | +| MobileNetV3_small_x0_35_ssld | 0.556 | 0.530 | 0.026 | 2.23 | 1.66 | 1.43 | 14.56 | 1.67 | 6.9 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_35_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_35_ssld_infer.tar) | +| MobileNetV3_large_x1_0_ssld | 0.790 | 0.753 | 0.036 | 16.55 | 10.09 | 6.84 | 229.66 | 5.50 | 21 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_0_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_ssld_infer.tar) | +| MobileNetV3_small_x1_0_ssld | 0.713 | 0.682 | 0.031 | 5.63 | 3.65 | 2.60 | 63.67 | 2.95 | 12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_0_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_0_ssld_infer.tar) | +| GhostNet_x1_3_ssld | 0.794 | 0.757 | 0.037 | 19.16 | 12.25 | 9.40 | 236.89 | 7.38 | 29 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_3_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_3_ssld_infer.tar) | + + + +### 2.3 Intel-CPU-side knowledge distillation model + +| Model | Top-1 Acc | Reference
Top-1 Acc | Acc gain | Intel-Xeon-Gold-6148 time(ms)
bs=1 | FLOPs(M) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|---------------------|-----------|-----------|---------------|----------------|----------|-----------|-----------------------------------|-----------------------------------| +| PPLCNet_x0_5_ssld | 0.661 | 0.631 | 0.030 | 2.05 | 47.28 | 1.89 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_5_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_5_ssld_infer.tar) | +| PPLCNet_x1_0_ssld | 0.744 | 0.713 | 0.033 | 2.46 | 160.81 | 2.96 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x1_0_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x1_0_ssld_infer.tar) | +| PPLCNet_x2_5_ssld | 0.808 | 0.766 | 0.042 | 5.39 | 906.49 | 9.04 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x2_5_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x2_5_ssld_infer.tar) | + + +* Note: `Reference Top-1 Acc` means the accuracy of the pre-trained model obtained by PaddleClas based on ImageNet1k dataset training. + + + +## 3. PP-LCNet series [[28](#ref28)] + +The accuracy and speed indicators of the PP-LCNet series models are shown in the following table. For more information about this series of models, please refer to: [PP-LCNet series model documents](../models/PP-LCNet_en.md)。 + +| Model | Top-1 Acc | Top-5 Acc | Intel-Xeon-Gold-6148 time(ms)
bs=1 | FLOPs(M) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|:--:|:--:|:--:|:--:|----|----|----|:--:| +| PPLCNet_x0_25 |0.5186 | 0.7565 | 1.61785 | 18.25 | 1.52 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_25_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_25_infer.tar) | +| PPLCNet_x0_35 |0.5809 | 0.8083 | 2.11344 | 29.46 | 1.65 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_35_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_35_infer.tar) | +| PPLCNet_x0_5 |0.6314 | 0.8466 | 2.72974 | 47.28 | 1.89 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_5_infer.tar) | +| PPLCNet_x0_75 |0.6818 | 0.8830 | 4.51216 | 98.82 | 2.37 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_75_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_75_infer.tar) | +| PPLCNet_x1_0 |0.7132 | 0.9003 | 6.49276 | 160.81 | 2.96 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x1_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x1_0_infer.tar) | +| PPLCNet_x1_5 |0.7371 | 0.9153 | 12.2601 | 341.86 | 4.52 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x1_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x1_5_infer.tar) | +| PPLCNet_x2_0 |0.7518 | 0.9227 | 20.1667 | 590 | 6.54 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x2_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x2_0_infer.tar) | +| PPLCNet_x2_5 |0.7660 | 0.9300 | 29.595 | 906 | 9.04 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x2_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x2_5_infer.tar) | + + + +## 4. ResNet series [[1](#ref1)] + +The accuracy and speed indicators of ResNet and ResNet_vd series models are shown in the following table. For more information about this series of models, please refer to: [ResNet and ResNet_vd series model documents](../models/ResNet_and_vd_en.md)。 + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|---------------------|-----------|-----------|-----------------------|----------------------|----------|-----------|----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------| +| ResNet18 | 0.7098 | 0.8992 | 1.22 | 2.19 | 3.63 | 1.83 | 11.70 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet18_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet18_infer.tar) | +| ResNet18_vd | 0.7226 | 0.9080 | 1.26 | 2.28 | 3.89 | 2.07 | 11.72 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet18_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet18_vd_infer.tar) | +| ResNet34 | 0.7457 | 0.9214 | 1.97 | 3.25 | 5.70 | 3.68 | 21.81 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_infer.tar) | +| ResNet34_vd | 0.7598 | 0.9298 | 2.00 | 3.28 | 5.84 | 3.93 | 21.84 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_vd_infer.tar) | +| ResNet34_vd_ssld | 0.7972 | 0.9490 | 2.00 | 3.28 | 5.84 | 3.93 | 21.84 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_vd_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_vd_ssld_infer.tar) | +| ResNet50 | 0.7650 | 0.9300 | 2.54 | 4.79 | 7.40 | 4.11 | 25.61 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_infer.tar) | +| ResNet50_vc | 0.7835 | 0.9403 | 2.57 | 4.83 | 7.52 | 4.35 | 25.63 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNet50_vc_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vc_infer.tar) | +| ResNet50_vd | 0.7912 | 0.9444 | 2.60 | 4.86 | 7.63 | 4.35 | 25.63 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar) | +| ResNet101 | 0.7756 | 0.9364 | 4.37 | 8.18 | 12.38 | 7.83 | 44.65 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_infer.tar) | +| ResNet101_vd | 0.8017 | 0.9497 | 4.43 | 8.25 | 12.60 | 8.08 | 44.67 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_vd_infer.tar) | +| ResNet152 | 0.7826 | 0.9396 | 6.05 | 11.41 | 17.33 | 11.56 | 60.34 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet152_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet152_infer.tar) | +| ResNet152_vd | 0.8059 | 0.9530 | 6.11 | 11.51 | 17.59 | 11.80 | 60.36 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet152_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet152_vd_infer.tar) | +| ResNet200_vd | 0.8093 | 0.9533 | 7.70 | 14.57 | 22.16 | 15.30 | 74.93 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet200_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet200_vd_infer.tar) | +| ResNet50_vd_
ssld | 0.8300 | 0.9640 | 2.60 | 4.86 | 7.63 | 4.35 | 25.63 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_ssld_infer.tar) | +| ResNet101_vd_
ssld | 0.8373 | 0.9669 | 4.43 | 8.25 | 12.60 | 8.08 | 44.67 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_vd_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_vd_ssld_infer.tar) | + + + +## 5. Mobile series [[3](#ref3)][[4](#ref4)][[5](#ref5)][[6](#ref6)][[23](#ref23)] + +The accuracy and speed indicators of the mobile series models are shown in the following table. For more information about this series, please refer to: [Mobile series model documents](../models/Mobile_en.md)。 + +| Model | Top-1 Acc | Top-5 Acc | SD855 time(ms)
bs=1, thread=1 | SD855 time(ms)
bs=1, thread=2 | SD855 time(ms)
bs=1, thread=4 | FLOPs(M) | Params(M) | Model大小(M) | Pretrained Model Download Address | Inference Model Download Address | +|----------------------------------|-----------|-----------|------------------------|----------|-----------|---------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------| +| MobileNetV1_
x0_25 | 0.5143 | 0.7546 | 2.88 | 1.82 | 1.26 | 43.56 | 0.48 | 1.9 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_x0_25_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_x0_25_infer.tar) | +| MobileNetV1_
x0_5 | 0.6352 | 0.8473 | 8.74 | 5.26 | 3.09 | 154.57 | 1.34 | 5.2 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_x0_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_x0_5_infer.tar) | +| MobileNetV1_
x0_75 | 0.6881 | 0.8823 | 17.84 | 10.61 | 6.21 | 333.00 | 2.60 | 10 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_x0_75_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_x0_75_infer.tar) | +| MobileNetV1 | 0.7099 | 0.8968 | 30.24 | 17.86 | 10.30 | 578.88 | 4.25 | 16 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_infer.tar) | +| MobileNetV1_
ssld | 0.7789 | 0.9394 | 30.24 | 17.86 | 10.30 | 578.88 | 4.25 | 16 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_ssld_infer.tar) | +| MobileNetV2_
x0_25 | 0.5321 | 0.7652 | 3.46 | 2.51 | 2.03 | 34.18 | 1.53 | 6.1 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x0_25_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x0_25_infer.tar) | +| MobileNetV2_
x0_5 | 0.6503 | 0.8572 | 7.69 | 4.92 | 3.57 | 99.48 | 1.98 | 7.8 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x0_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x0_5_infer.tar) | +| MobileNetV2_
x0_75 | 0.6983 | 0.8901 | 13.69 | 8.60 | 5.82 | 197.37 | 2.65 | 10 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x0_75_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x0_75_infer.tar) | +| MobileNetV2 | 0.7215 | 0.9065 | 20.74 | 12.71 | 8.10 | 327.84 | 3.54 | 14 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_infer.tar) | +| MobileNetV2_
x1_5 | 0.7412 | 0.9167 | 40.79 | 24.49 | 15.50 | 702.35 | 6.90 | 26 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x1_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x1_5_infer.tar) | +| MobileNetV2_
x2_0 | 0.7523 | 0.9258 | 67.50 | 40.03 | 25.55 | 1217.25 | 11.33 | 43 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x2_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x2_0_infer.tar) | +| MobileNetV2_
ssld | 0.7674 | 0.9339 | 20.74 | 12.71 | 8.10 | 327.84 | 3.54 | 14 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_ssld_infer.tar) | +| MobileNetV3_
large_x1_25 | 0.7641 | 0.9295 | 24.52 | 14.76 | 9.89 | 362.70 | 7.47 | 29 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_25_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_25_infer.tar) | +| MobileNetV3_
large_x1_0 | 0.7532 | 0.9231 | 16.55 | 10.09 | 6.84 | 229.66 | 5.50 | 21 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_infer.tar) | +| MobileNetV3_
large_x0_75 | 0.7314 | 0.9108 | 11.53 | 7.06 | 4.94 | 151.70 | 3.93 | 16 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x0_75_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x0_75_infer.tar) | +| MobileNetV3_
large_x0_5 | 0.6924 | 0.8852 | 6.50 | 4.22 | 3.15 | 71.83 | 2.69 | 11 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x0_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x0_5_infer.tar) | +| MobileNetV3_
large_x0_35 | 0.6432 | 0.8546 | 4.43 | 3.11 | 2.41 | 40.90 | 2.11 | 8.6 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x0_35_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x0_35_infer.tar) | +| MobileNetV3_
small_x1_25 | 0.7067 | 0.8951 | 7.88 | 4.91 | 3.45 | 100.07 | 3.64 | 14 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_25_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_25_infer.tar) | +| MobileNetV3_
small_x1_0 | 0.6824 | 0.8806 | 5.63 | 3.65 | 2.60 | 63.67 | 2.95 | 12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_0_infer.tar) | +| MobileNetV3_
small_x0_75 | 0.6602 | 0.8633 | 4.50 | 2.96 | 2.19 | 46.02 | 2.38 | 9.6 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_75_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_75_infer.tar) | +| MobileNetV3_
small_x0_5 | 0.5921 | 0.8152 | 2.89 | 2.04 | 1.62 | 22.60 | 1.91 | 7.8 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_5_infer.tar) | +| MobileNetV3_
small_x0_35 | 0.5303 | 0.7637 | 2.23 | 1.66 | 1.43 | 14.56 | 1.67 | 6.9 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_35_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_35_infer.tar) | +| MobileNetV3_
small_x0_35_ssld | 0.5555 | 0.7771 | 2.23 | 1.66 | 1.43 | 14.56 | 1.67 | 6.9 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_35_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_35_ssld_infer.tar) | +| MobileNetV3_
large_x1_0_ssld | 0.7896 | 0.9448 | 16.55 | 10.09 | 6.84 | 229.66 | 5.50 | 21 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_0_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_ssld_infer.tar) | +| MobileNetV3_small_
x1_0_ssld | 0.7129 | 0.9010 | 5.63 | 3.65 | 2.60 | 63.67 | 2.95 | 12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_0_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_0_ssld_infer.tar) | +| ShuffleNetV2 | 0.6880 | 0.8845 | 9.72 | 5.97 | 4.13 | 148.86 | 2.29 | 9 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x1_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x1_0_infer.tar) | +| ShuffleNetV2_
x0_25 | 0.4990 | 0.7379 | 1.94 | 1.53 | 1.43 | 18.95 | 0.61 | 2.7 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x0_25_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x0_25_infer.tar) | +| ShuffleNetV2_
x0_33 | 0.5373 | 0.7705 | 2.23 | 1.70 | 1.79 | 24.04 | 0.65 | 2.8 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x0_33_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x0_33_infer.tar) | +| ShuffleNetV2_
x0_5 | 0.6032 | 0.8226 | 3.67 | 2.63 | 2.06 | 42.58 | 1.37 | 5.6 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x0_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x0_5_infer.tar) | +| ShuffleNetV2_
x1_5 | 0.7163 | 0.9015 | 17.21 | 10.56 | 6.81 | 301.35 | 3.53 | 14 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x1_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x1_5_infer.tar) | +| ShuffleNetV2_
x2_0 | 0.7315 | 0.9120 | 31.21 | 18.98 | 11.65 | 571.70 | 7.40 | 28 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x2_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x2_0_infer.tar) | +| ShuffleNetV2_
swish | 0.7003 | 0.8917 | 31.21 | 9.06 | 5.74 | 148.86 | 2.29 | 9.1 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_swish_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_swish_infer.tar) | +| GhostNet_
x0_5 | 0.6688 | 0.8695 | 5.28 | 3.95 | 3.29 | 46.15 | 2.60 | 10 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x0_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x0_5_infer.tar) | +| GhostNet_
x1_0 | 0.7402 | 0.9165 | 12.89 | 8.66 | 6.72 | 148.78 | 5.21 | 20 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_0_infer.tar) | +| GhostNet_
x1_3 | 0.7579 | 0.9254 | 19.16 | 12.25 | 9.40 | 236.89 | 7.38 | 29 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_3_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_3_infer.tar) | +| GhostNet_
x1_3_ssld | 0.7938 | 0.9449 | 19.16 | 12.25 | 9.40 | 236.89 | 7.38 | 29 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_3_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_3_ssld_infer.tar) | +| ESNet_x0_25 | 0.6248 | 0.8346 |4.12|2.97|2.51| 30.85 | 2.83 | 11 |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x0_25_pretrained.pdparams) |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x0_25_infer.tar) | +| ESNet_x0_5 | 0.6882 | 0.8804 |6.45|4.42|3.35| 67.31 | 3.25 | 13 |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x0_5_pretrained.pdparams) |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x0_5_infer.tar) | +| ESNet_x0_75 | 0.7224 | 0.9045 |9.59|6.28|4.52| 123.74 | 3.87 | 15 |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x0_75_pretrained.pdparams) |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x0_75_infer.tar) | +| ESNet_x1_0 | 0.7392 | 0.9140 |13.67|8.71|5.97| 197.33 | 4.64 | 18 |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x1_0_pretrained.pdparams) |[Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x1_0_infer.tar) | + + + +## 6. SEResNeXt and Res2Net series [[7](#ref7)][[8](#ref8)][[9](#ref9)] + +The accuracy and speed indicators of the SEResNeXt and Res2Net series models are shown in the following table. For more information about the models of this series, please refer to: [SEResNeXt and Res2Net series model documents](../models/SEResNext_and_Res2Net_en.md). + + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|---------------------------|-----------|-----------|-----------------------|----------------------|----------|-----------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------| +| Res2Net50_
26w_4s | 0.7933 | 0.9457 | 3.52 | 6.23 | 9.30 | 4.28 | 25.76 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_26w_4s_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_26w_4s_infer.tar) | +| Res2Net50_vd_
26w_4s | 0.7975 | 0.9491 | 3.59 | 6.35 | 9.50 | 4.52 | 25.78 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_vd_26w_4s_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_vd_26w_4s_infer.tar) | +| Res2Net50_
14w_8s | 0.7946 | 0.9470 | 4.39 | 7.21 | 10.38 | 4.20 | 25.12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_14w_8s_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_14w_8s_infer.tar) | +| Res2Net101_vd_
26w_4s | 0.8064 | 0.9522 | 6.34 | 11.02 | 16.13 | 8.35 | 45.35 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net101_vd_26w_4s_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net101_vd_26w_4s_infer.tar) | +| Res2Net200_vd_
26w_4s | 0.8121 | 0.9571 | 11.45 | 19.77 | 28.81 | 15.77 | 76.44 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net200_vd_26w_4s_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net200_vd_26w_4s_infer.tar) | +| Res2Net200_vd_
26w_4s_ssld | 0.8513 | 0.9742 | 11.45 | 19.77 | 28.81 | 15.77 | 76.44 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net200_vd_26w_4s_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net200_vd_26w_4s_ssld_infer.tar) | +| ResNeXt50_
32x4d | 0.7775 | 0.9382 | 5.07 | 8.49 | 12.02 | 4.26 | 25.10 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_32x4d_infer.tar) | +| ResNeXt50_vd_
32x4d | 0.7956 | 0.9462 | 5.29 | 8.68 | 12.33 | 4.50 | 25.12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_vd_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_vd_32x4d_infer.tar) | +| ResNeXt50_
64x4d | 0.7843 | 0.9413 | 9.39 | 13.97 | 20.56 | 8.02 | 45.29 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_64x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_64x4d_infer.tar) | +| ResNeXt50_vd_
64x4d | 0.8012 | 0.9486 | 9.75 | 14.14 | 20.84 | 8.26 | 45.31 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_vd_64x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_vd_64x4d_infer.tar) | +| ResNeXt101_
32x4d | 0.7865 | 0.9419 | 11.34 | 16.78 | 22.80 | 8.01 | 44.32 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x4d_infer.tar) | +| ResNeXt101_vd_
32x4d | 0.8033 | 0.9512 | 11.36 | 17.01 | 23.07 | 8.25 | 44.33 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_vd_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_vd_32x4d_infer.tar) | +| ResNeXt101_
64x4d | 0.7835 | 0.9452 | 21.57 | 28.08 | 39.49 | 15.52 | 83.66 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_64x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_64x4d_infer.tar) | +| ResNeXt101_vd_
64x4d | 0.8078 | 0.9520 | 21.57 | 28.22 | 39.70 | 15.76 | 83.68 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_vd_64x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_vd_64x4d_infer.tar) | +| ResNeXt152_
32x4d | 0.7898 | 0.9433 | 17.14 | 25.11 | 33.79 | 11.76 | 60.15 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_32x4d_infer.tar) | +| ResNeXt152_vd_
32x4d | 0.8072 | 0.9520 | 16.99 | 25.29 | 33.85 | 12.01 | 60.17 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_vd_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_vd_32x4d_infer.tar) | +| ResNeXt152_
64x4d | 0.7951 | 0.9471 | 33.07 | 42.05 | 59.13 | 23.03 | 115.27 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_64x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_64x4d_infer.tar) | +| ResNeXt152_vd_
64x4d | 0.8108 | 0.9534 | 33.30 | 42.41 | 59.42 | 23.27 | 115.29 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_vd_64x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_vd_64x4d_infer.tar) | +| SE_ResNet18_vd | 0.7333 | 0.9138 | 1.48 | 2.70 | 4.32 | 2.07 | 11.81 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNet18_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNet18_vd_infer.tar) | +| SE_ResNet34_vd | 0.7651 | 0.9320 | 2.42 | 3.69 | 6.29 | 3.93 | 22.00 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNet34_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNet34_vd_infer.tar) | +| SE_ResNet50_vd | 0.7952 | 0.9475 | 3.11 | 5.99 | 9.34 | 4.36 | 28.16 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNet50_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNet50_vd_infer.tar) | +| SE_ResNeXt50_
32x4d | 0.7844 | 0.9396 | 6.39 | 11.01 | 14.94 | 4.27 | 27.63 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNeXt50_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNeXt50_32x4d_infer.tar) | +| SE_ResNeXt50_vd_
32x4d | 0.8024 | 0.9489 | 7.04 | 11.57 | 16.01 | 5.64 | 27.76 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNeXt50_vd_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNeXt50_vd_32x4d_infer.tar) | +| SE_ResNeXt101_
32x4d | 0.7939 | 0.9443 | 13.31 | 21.85 | 28.77 | 8.03 | 49.09 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNeXt101_32x4d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNeXt101_32x4d_infer.tar) | +| SENet154_vd | 0.8140 | 0.9548 | 34.83 | 51.22 | 69.74 | 24.45 | 122.03 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SENet154_vd_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SENet154_vd_infer.tar) | + + + +## 7. DPN and DenseNet series [[14](#ref14)][[15](#ref15)] + +The accuracy and speed indicators of the DPN and DenseNet series models are shown in the following table. For more information about the models of this series, please refer to: [DPN and DenseNet series model documents](../models/DPN_DenseNet_en.md). + + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|-------------|-----------|-----------|-----------------------|----------------------|----------|-----------|--------------------------------------------------------------------------------------|-------------|-------------| +| DenseNet121 | 0.7566 | 0.9258 | 3.40 | 6.94 | 9.17 | 2.87 | 8.06 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet121_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet121_infer.tar) | +| DenseNet161 | 0.7857 | 0.9414 | 7.06 | 14.37 | 19.55 | 7.79 | 28.90 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet161_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet161_infer.tar) | +| DenseNet169 | 0.7681 | 0.9331 | 5.00 | 10.29 | 12.84 | 3.40 | 14.31 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet169_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet169_infer.tar) | +| DenseNet201 | 0.7763 | 0.9366 | 6.38 | 13.72 | 17.17 | 4.34 | 20.24 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet201_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet201_infer.tar) | +| DenseNet264 | 0.7796 | 0.9385 | 9.34 | 20.95 | 25.41 | 5.82 | 33.74 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet264_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet264_infer.tar) | +| DPN68 | 0.7678 | 0.9343 | 8.18 | 11.40 | 14.82 | 2.35 | 12.68 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN68_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN68_infer.tar) | +| DPN92 | 0.7985 | 0.9480 | 12.48 | 20.04 | 25.10 | 6.54 | 37.79 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN92_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN92_infer.tar) | +| DPN98 | 0.8059 | 0.9510 | 14.70 | 25.55 | 35.12 | 11.728 | 61.74 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN98_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN98_infer.tar) | +| DPN107 | 0.8089 | 0.9532 | 19.46 | 35.62 | 50.22 | 18.38 | 87.13 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN107_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN107_infer.tar) | +| DPN131 | 0.8070 | 0.9514 | 19.64 | 34.60 | 47.42 | 16.09 | 79.48 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN131_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN131_infer.tar) | + + + +## 8. HRNet series [[13](#ref13)] + +The accuracy and speed indicators of the HRNet series models are shown in the following table. For more information about the models of this series, please refer to: [HRNet series model documents](../models/HRNet_en.md). + + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|-------------|-----------|-----------|------------------|------------------|----------|-----------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------| +| HRNet_W18_C | 0.7692 | 0.9339 | 6.66 | 8.94 | 11.95 | 4.32 | 21.35 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W18_C_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W18_C_infer.tar) | +| HRNet_W18_C_ssld | 0.81162 | 0.95804 | 6.66 | 8.94 | 11.95 | 4.32 | 21.35 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W18_C_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W18_C_ssld_infer.tar) | +| HRNet_W30_C | 0.7804 | 0.9402 | 8.61 | 11.40 | 15.23 | 8.15 | 37.78 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W30_C_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W30_C_infer.tar) | +| HRNet_W32_C | 0.7828 | 0.9424 | 8.54 | 11.58 | 15.57 | 8.97 | 41.30 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W32_C_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W32_C_infer.tar) | +| HRNet_W40_C | 0.7877 | 0.9447 | 9.83 | 15.02 | 20.92 | 12.74 | 57.64 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W40_C_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W40_C_infer.tar) | +| HRNet_W44_C | 0.7900 | 0.9451 | 10.62 | 16.18 | 25.92 | 14.94 | 67.16 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W44_C_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W44_C_infer.tar) | +| HRNet_W48_C | 0.7895 | 0.9442 | 11.07 | 17.06 | 27.28 | 17.34 | 77.57 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W48_C_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W48_C_infer.tar) | +| HRNet_W48_C_ssld | 0.8363 | 0.9682 | 11.07 | 17.06 | 27.28 | 17.34 | 77.57 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W48_C_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W48_C_ssld_infer.tar) | +| HRNet_W64_C | 0.7930 | 0.9461 | 13.82 | 21.15 | 35.51 | 28.97 | 128.18 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W64_C_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W64_C_infer.tar) | +| SE_HRNet_W64_C_ssld | 0.8475 | 0.9726 | 17.11 | 26.87 | 43.24 | 29.00 | 129.12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/SE_HRNet_W64_C_ssld_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_HRNet_W64_C_ssld_infer.tar) | + + + +## 9. Inception series [[10](#ref10)][[11](#ref11)][[12](#ref12)][[26](#ref26)] + +The accuracy and speed indicators of the Inception series models are shown in the following table. For more information about this series of models, please refer to: [Inception series model documents](../models/Inception_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|--------------------|-----------|-----------|-----------------------|----------------------|----------|-----------|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------| +| GoogLeNet | 0.7070 | 0.8966 | 1.41 | 3.25 | 5.00 | 1.44 | 11.54 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GoogLeNet_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GoogLeNet_infer.tar) | +| Xception41 | 0.7930 | 0.9453 | 3.58 | 8.76 | 16.61 | 8.57 | 23.02 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception41_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception41_infer.tar) | +| Xception41_deeplab | 0.7955 | 0.9438 | 3.81 | 9.16 | 17.20 | 9.28 | 27.08 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception41_deeplab_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception41_deeplab_infer.tar) | +| Xception65 | 0.8100 | 0.9549 | 5.45 | 12.78 | 24.53 | 13.25 | 36.04 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception65_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception65_infer.tar) | +| Xception65_deeplab | 0.8032 | 0.9449 | 5.65 | 13.08 | 24.61 | 13.96 | 40.10 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception65_deeplab_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception65_deeplab_infer.tar) | +| Xception71 | 0.8111 | 0.9545 | 6.19 | 15.34 | 29.21 | 16.21 | 37.86 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception71_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception71_infer.tar) | +| InceptionV3 | 0.7914 | 0.9459 | 4.78 | 8.53 | 12.28 | 5.73 | 23.87 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/InceptionV3_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/InceptionV3_infer.tar) | +| InceptionV4 | 0.8077 | 0.9526 | 8.93 | 15.17 | 21.56 | 12.29 | 42.74 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/InceptionV4_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/InceptionV4_infer.tar) | + + + +## 10. EfficientNet and ResNeXt101_wsl series [[16](#ref16)][[17](#ref17)] + +The accuracy and speed indicators of the EfficientNet and ResNeXt101_wsl series models are shown in the following table. For more information about this series of models, please refer to: [EfficientNet and ResNeXt101_wsl series model documents](../models/EfficientNet_and_ResNeXt101_wsl_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|---------------------------|-----------|-----------|------------------|------------------|----------|-----------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------| +| ResNeXt101_
32x8d_wsl | 0.8255 | 0.9674 | 13.55 | 23.39 | 36.18 | 16.48 | 88.99 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x8d_wsl_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x8d_wsl_infer.tar) | +| ResNeXt101_
32x16d_wsl | 0.8424 | 0.9726 | 21.96 | 38.35 | 63.29 | 36.26 | 194.36 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x16d_wsl_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x16d_wsl_infer.tar) | +| ResNeXt101_
32x32d_wsl | 0.8497 | 0.9759 | 37.28 | 76.50 | 121.56 | 87.28 | 469.12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x32d_wsl_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x32d_wsl_infer.tar) | +| ResNeXt101_
32x48d_wsl | 0.8537 | 0.9769 | 55.07 | 124.39 | 205.01 | 153.57 | 829.26 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x48d_wsl_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x48d_wsl_infer.tar) | +| Fix_ResNeXt101_
32x48d_wsl | 0.8626 | 0.9797 | 55.01 | 122.63 | 204.66 | 313.41 | 829.26 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Fix_ResNeXt101_32x48d_wsl_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Fix_ResNeXt101_32x48d_wsl_infer.tar) | +| EfficientNetB0 | 0.7738 | 0.9331 | 1.96 | 3.71 | 5.56 | 0.40 | 5.33 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB0_infer.tar) | +| EfficientNetB1 | 0.7915 | 0.9441 | 2.88 | 5.40 | 7.63 | 0.71 | 7.86 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB1_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB1_infer.tar) | +| EfficientNetB2 | 0.7985 | 0.9474 | 3.26 | 6.20 | 9.17 | 1.02 | 9.18 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB2_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB2_infer.tar) | +| EfficientNetB3 | 0.8115 | 0.9541 | 4.52 | 8.85 | 13.54 | 1.88 | 12.324 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB3_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB3_infer.tar) | +| EfficientNetB4 | 0.8285 | 0.9623 | 6.78 | 15.47 | 24.95 | 4.51 | 19.47 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB4_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB4_infer.tar) | +| EfficientNetB5 | 0.8362 | 0.9672 | 10.97 | 27.24 | 45.93 | 10.51 | 30.56 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB5_infer.tar) | +| EfficientNetB6 | 0.8400 | 0.9688 | 17.09 | 43.32 | 76.90 | 19.47 | 43.27 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB6_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB6_infer.tar) | +| EfficientNetB7 | 0.8430 | 0.9689 | 25.91 | 71.23 | 128.20 | 38.45 | 66.66 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB7_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB7_infer.tar) | +| EfficientNetB0_
small | 0.7580 | 0.9258 | 1.24 | 2.59 | 3.92 | 0.40 | 4.69 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB0_small_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB0_small_infer.tar) | + + + +## 11. ResNeSt and RegNet series [[24](#ref24)][[25](#ref25)] + +The accuracy and speed indicators of the ResNeSt and RegNet series models are shown in the following table. For more information about the models of this series, please refer to: [ResNeSt and RegNet series model documents](../models/ResNeSt_RegNet_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|------------------------|-----------|-----------|------------------|------------------|----------|-----------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| +| ResNeSt50_
fast_1s1x64d | 0.8035 | 0.9528 | 2.73 | 5.33 | 8.24 | 4.36 | 26.27 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeSt50_fast_1s1x64d_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeSt50_fast_1s1x64d_infer.tar) | +| ResNeSt50 | 0.8083 | 0.9542 | 7.36 | 10.23 | 13.84 | 5.40 | 27.54 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeSt50_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeSt50_infer.tar) | +| RegNetX_4GF | 0.785 | 0.9416 | 6.46 | 8.48 | 11.45 | 4.00 | 22.23 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RegNetX_4GF_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RegNetX_4GF_infer.tar) | + + + +## 12. ViT and DeiT series [[31](#ref31)][[32](#ref32)] + +The accuracy and speed indicators of ViT (Vision Transformer) and DeiT (Data-efficient Image Transformers) series models are shown in the following table. For more information about this series of models, please refer to: [ViT_and_DeiT series model documents](../models/ViT_and_DeiT_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|------------------------|-----------|-----------|------------------|------------------|----------|------------------------|------------------------|------------------------|------------------------| +| ViT_small_
patch16_224 | 0.7769 | 0.9342 | 3.71 | 9.05 | 16.72 | 9.41 | 48.60 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_small_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_small_patch16_224_infer.tar) | +| ViT_base_
patch16_224 | 0.8195 | 0.9617 | 6.12 | 14.84 | 28.51 | 16.85 | 86.42 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_base_patch16_224_infer.tar) | +| ViT_base_
patch16_384 | 0.8414 | 0.9717 | 14.15 | 48.38 | 95.06 | 49.35 | 86.42 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch16_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_base_patch16_384_infer.tar) | +| ViT_base_
patch32_384 | 0.8176 | 0.9613 | 4.94 | 13.43 | 24.08 | 12.66 | 88.19 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch32_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_base_patch32_384_infer.tar) | +| ViT_large_
patch16_224 | 0.8323 | 0.9650 | 15.53 | 49.50 | 94.09 | 59.65 | 304.12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_large_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_large_patch16_224_infer.tar) | +|ViT_large_
patch16_384| 0.8513 | 0.9736 | 39.51 | 152.46 | 304.06 | 174.70 | 304.12 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_large_patch16_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_large_patch16_384_infer.tar) | +|ViT_large_
patch32_384| 0.8153 | 0.9608 | 11.44 | 36.09 | 70.63 | 44.24 | 306.48 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_large_patch32_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_large_patch32_384_infer.tar) | + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|------------------------|-----------|-----------|------------------|------------------|----------|------------------------|------------------------|------------------------|------------------------| +| DeiT_tiny_
patch16_224 | 0.718 | 0.910 | 3.61 | 3.94 | 6.10 | 1.07 | 5.68 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_tiny_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_tiny_patch16_224_infer.tar) | +| DeiT_small_
patch16_224 | 0.796 | 0.949 | 3.61 | 6.24 | 10.49 | 4.24 | 21.97 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_small_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_small_patch16_224_infer.tar) | +| DeiT_base_
patch16_224 | 0.817 | 0.957 | 6.13 | 14.87 | 28.50 | 16.85 | 86.42 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_patch16_224_infer.tar) | +| DeiT_base_
patch16_384 | 0.830 | 0.962 | 14.12 | 48.80 | 97.60 | 49.35 | 86.42 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_patch16_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_patch16_384_infer.tar) | +| DeiT_tiny_
distilled_patch16_224 | 0.741 | 0.918 | 3.51 | 4.05 | 6.03 | 1.08 | 5.87 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_tiny_distilled_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_tiny_distilled_patch16_224_infer.tar) | +| DeiT_small_
distilled_patch16_224 | 0.809 | 0.953 | 3.70 | 6.20 | 10.53 | 4.26 | 22.36 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_small_distilled_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_small_distilled_patch16_224_infer.tar) | +| DeiT_base_
distilled_patch16_224 | 0.831 | 0.964 | 6.17 | 14.94 | 28.58 | 16.93 | 87.18 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_distilled_patch16_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_distilled_patch16_224_infer.tar) | +| DeiT_base_
distilled_patch16_384 | 0.851 | 0.973 | 14.12 | 48.76 | 97.09 | 49.43 | 87.18 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_distilled_patch16_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_distilled_patch16_384_infer.tar) | + + + +## 13. RepVGG series [[36](#ref36)] + +The accuracy and speed indicators of RepVGG series models are shown in the following table. For more introduction, please refer to: [RepVGG series model documents](../models/RepVGG_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|------------------------|-----------|-----------|------------------|------------------|----------|-----------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| +| RepVGG_A0 | 0.7131 | 0.9016 | | | | 1.36 | 8.31 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_A0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_A0_infer.tar) | +| RepVGG_A1 | 0.7380 | 0.9146 | | | | 2.37 | 12.79 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_A1_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_A1_infer.tar) | +| RepVGG_A2 | 0.7571 | 0.9264 | | | | 5.12 | 25.50 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_A2_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_A2_infer.tar) | +| RepVGG_B0 | 0.7450 | 0.9213 | | | | 3.06 | 14.34 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B0_infer.tar) | +| RepVGG_B1 | 0.7773 | 0.9385 | | | | 11.82 | 51.83 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B1_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B1_infer.tar) | +| RepVGG_B2 | 0.7813 | 0.9410 | | | | 18.38 | 80.32 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B2_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B2_infer.tar) | +| RepVGG_B1g2 | 0.7732 | 0.9359 | | | | 8.82 | 41.36 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B1g2_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B1g2_infer.tar) | +| RepVGG_B1g4 | 0.7675 | 0.9335 | | | | 7.31 | 36.13 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B1g4_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B1g4_infer.tar) | +| RepVGG_B2g4 | 0.7881 | 0.9448 | | | | 11.34 | 55.78 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B2g4_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B2g4_infer.tar) | +| RepVGG_B3g4 | 0.7965 | 0.9485 | | | | 16.07 | 75.63 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B3g4_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B3g4_infer.tar) | + + + +## 14. MixNet series [[29](#ref29)] + +The accuracy and speed indicators of the MixNet series models are shown in the following table. For more introduction, please refer to: [MixNet series model documents](../models/MixNet_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(M) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| -------- | --------- | --------- | ---------------- | ---------------- | ----------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| MixNet_S | 0.7628 | 0.9299 | 2.31 | 3.63 | 5.20 | 252.977 | 4.167 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MixNet_S_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MixNet_S_infer.tar) | +| MixNet_M | 0.7767 | 0.9364 | 2.84 | 4.60 | 6.62 | 357.119 | 5.065 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MixNet_M_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MixNet_M_infer.tar) | +| MixNet_L | 0.7860 | 0.9437 | 3.16 | 5.55 | 8.03 | 579.017 | 7.384 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MixNet_L_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MixNet_L_infer.tar) | + + + +## 15. ReXNet series [[30](#ref30)] + +The accuracy and speed indicators of ReXNet series models are shown in the following table. For more introduction, please refer to: [ReXNet series model documents](../models/ReXNet_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| ReXNet_1_0 | 0.7746 | 0.9370 | 3.08 | 4.15 | 5.49 | 0.415 | 4.84 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_1_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_1_0_infer.tar) | +| ReXNet_1_3 | 0.7913 | 0.9464 | 3.54 | 4.87 | 6.54 | 0.68 | 7.61 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_1_3_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_1_3_infer.tar) | +| ReXNet_1_5 | 0.8006 | 0.9512 | 3.68 | 5.31 | 7.38 | 0.90 | 9.79 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_1_5_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_1_5_infer.tar) | +| ReXNet_2_0 | 0.8122 | 0.9536 | 4.30 | 6.54 | 9.19 | 1.56 | 16.45 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_2_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_2_0_infer.tar) | +| ReXNet_3_0 | 0.8209 | 0.9612 | 5.74 | 9.49 | 13.62 | 3.44 | 34.83 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_3_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_3_0_infer.tar) | + + + +## 16. SwinTransformer series [[27](#ref27)] + +The accuracy and speed indicators of SwinTransformer series models are shown in the following table. For more introduction, please refer to: [SwinTransformer series model documents](../models/SwinTransformer_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| SwinTransformer_tiny_patch4_window7_224 | 0.8069 | 0.9534 | 6.59 | 9.68 | 16.32 | 4.35 | 28.26 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_tiny_patch4_window7_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_tiny_patch4_window7_224_infer.tar) | +| SwinTransformer_small_patch4_window7_224 | 0.8275 | 0.9613 | 12.54 | 17.07 | 28.08 | 8.51 | 49.56 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_small_patch4_window7_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_small_patch4_window7_224_infer.tar) | +| SwinTransformer_base_patch4_window7_224 | 0.8300 | 0.9626 | 13.37 | 23.53 | 39.11 | 15.13 | 87.70 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window7_224_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window7_224_infer.tar) | +| SwinTransformer_base_patch4_window12_384 | 0.8439 | 0.9693 | 19.52 | 64.56 | 123.30 | 44.45 | 87.70 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window12_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window12_384_infer.tar) | +| SwinTransformer_base_patch4_window7_224[1] | 0.8487 | 0.9746 | 13.53 | 23.46 | 39.13 | 15.13 | 87.70 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window7_224_22kto1k_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window7_224_infer.tar) | +| SwinTransformer_base_patch4_window12_384[1] | 0.8642 | 0.9807 | 19.65 | 64.72 | 123.42 | 44.45 | 87.70 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window12_384_22kto1k_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window12_384_infer.tar) | +| SwinTransformer_large_patch4_window7_224[1] | 0.8596 | 0.9783 | 15.74 | 38.57 | 71.49 | 34.02 | 196.43 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_large_patch4_window7_224_22kto1k_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_large_patch4_window7_224_22kto1k_infer.tar) | +| SwinTransformer_large_patch4_window12_384[1] | 0.8719 | 0.9823 | 32.61 | 116.59 | 223.23 | 99.97 | 196.43 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_large_patch4_window12_384_22kto1k_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_large_patch4_window12_384_22kto1k_infer.tar) | + +[1]：It is pre-trained based on the ImageNet22k dataset, and then transferred and learned from the ImageNet1k dataset. + + + +## 17. LeViT series [[33](#ref33)] + +The accuracy and speed indicators of LeViT series models are shown in the following table. For more introduction, please refer to: [LeViT series model documents](../models/LeViT_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(M) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| LeViT_128S | 0.7598 | 0.9269 | | | | 281 | 7.42 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_128S_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_128S_infer.tar) | +| LeViT_128 | 0.7810 | 0.9371 | | | | 365 | 8.87 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_128_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_128_infer.tar) | +| LeViT_192 | 0.7934 | 0.9446 | | | | 597 | 10.61 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_192_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_192_infer.tar) | +| LeViT_256 | 0.8085 | 0.9497 | | | | 1049 | 18.45 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_256_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_256_infer.tar) | +| LeViT_384 | 0.8191 | 0.9551 | | | | 2234 | 38.45 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_384_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_384_infer.tar) | + +**Note**: The accuracy difference with Reference is due to the difference in data preprocessing and the use of no distilled head as output. + + + +## 18. Twins series [[34](#ref34)] + +The accuracy and speed indicators of Twins series models are shown in the following table. For more introduction, please refer to: [Twins series model documents](../models/Twins_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| pcpvt_small | 0.8082 | 0.9552 | 7.32 | 10.51 | 15.27 |3.67 | 24.06 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/pcpvt_small_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/pcpvt_small_infer.tar) | +| pcpvt_base | 0.8242 | 0.9619 | 12.20 | 16.22 | 23.16 | 6.44 | 43.83 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/pcpvt_base_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/pcpvt_base_infer.tar) | +| pcpvt_large | 0.8273 | 0.9650 | 16.47 | 22.90 | 32.73 | 9.50 | 60.99 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/pcpvt_large_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/pcpvt_large_infer.tar) | +| alt_gvt_small | 0.8140 | 0.9546 | 6.94 | 9.01 | 12.27 |2.81 | 24.06 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/alt_gvt_small_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/alt_gvt_small_infer.tar) | +| alt_gvt_base | 0.8294 | 0.9621 | 9.37 | 15.02 | 24.54 | 8.34 | 56.07 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/alt_gvt_base_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/alt_gvt_base_infer.tar) | +| alt_gvt_large | 0.8331 | 0.9642 | 11.76 | 22.08 | 35.12 | 14.81 | 99.27 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/alt_gvt_large_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/alt_gvt_large_infer.tar) | + +**Note**: The accuracy difference with Reference is due to the difference in data preprocessing. + + + +## 19. HarDNet series [[37](#ref37)] + +The accuracy and speed indicators of HarDNet series models are shown in the following table. For more introduction, please refer to: [HarDNet series model documents](../models/HarDNet_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| HarDNet39_ds | 0.7133 |0.8998 | 1.40 | 2.30 | 3.33 | 0.44 | 3.51 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet39_ds_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet39_ds_infer.tar) | +| HarDNet68_ds |0.7362 | 0.9152 | 2.26 | 3.34 | 5.06 | 0.79 | 4.20 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet68_ds_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet68_ds_infer.tar) | +| HarDNet68| 0.7546 | 0.9265 | 3.58 | 8.53 | 11.58 | 4.26 | 17.58 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet68_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet68_infer.tar) | +| HarDNet85 | 0.7744 | 0.9355 | 6.24 | 14.85 | 20.57 | 9.09 | 36.69 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet85_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet85_infer.tar) | + + + +## 20. DLA series [[38](#ref38)] + +The accuracy and speed indicators of DLA series models are shown in the following table. For more introduction, please refer to: [DLA series model documents](../models/DLA_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| DLA102 | 0.7893 |0.9452 | 4.95 | 8.08 | 12.40 | 7.19 | 33.34 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA102_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA102_infer.tar) | +| DLA102x2 |0.7885 | 0.9445 | 19.58 | 23.97 | 31.37 | 9.34 | 41.42 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA102x2_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA102x2_infer.tar) | +| DLA102x| 0.781 | 0.9400 | 11.12 | 15.60 | 20.37 | 5.89 | 26.40 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA102x_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA102x_infer.tar) | +| DLA169 | 0.7809 | 0.9409 | 7.70 | 12.25 | 18.90 | 11.59 | 53.50 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA169_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA169_infer.tar) | +| DLA34 | 0.7603 | 0.9298 | 1.83 | 3.37 | 5.98 | 3.07 | 15.76 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA34_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA34_infer.tar) | +| DLA46_c |0.6321 | 0.853 | 1.06 | 2.08 | 3.23 | 0.54 | 1.31 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA46_c_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA46_c_infer.tar) | +| DLA60 | 0.7610 | 0.9292 | 2.78 | 5.36 | 8.29 | 4.26 | 22.08 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA60_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA60_infer.tar) | +| DLA60x_c | 0.6645 | 0.8754 | 1.79 | 3.68 | 5.19 | 0.59 | 1.33 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA60x_c_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA60x_c_infer.tar) | +| DLA60x | 0.7753 | 0.9378 | 5.98 | 9.24 | 12.52 | 3.54 | 17.41 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA60x_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA60x_infer.tar) | + + + +## 21. RedNet series [[39](#ref39)] + +The accuracy and speed indicators of RedNet series models are shown in the following table. For more introduction, please refer to: [RedNet series model documents](../models/RedNet_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| RedNet26 | 0.7595 |0.9319 | 4.45 | 15.16 | 29.03 | 1.69 | 9.26 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet26_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet26_infer.tar) | +| RedNet38 |0.7747 | 0.9356 | 6.24 | 21.39 | 41.26 | 2.14 | 12.43 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet38_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet38_infer.tar) | +| RedNet50| 0.7833 | 0.9417 | 8.04 | 27.71 | 53.73 | 2.61 | 15.60 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet50_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet50_infer.tar) | +| RedNet101 | 0.7894 | 0.9436 | 13.07 | 44.12 | 83.28 | 4.59 | 25.76 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet101_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet101_infer.tar) | +| RedNet152 | 0.7917 | 0.9440 | 18.66 | 63.27 | 119.48 | 6.57 | 34.14 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet152_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet152_infer.tar) | + + + +## 22. TNT series [[35](#ref35)] + +The accuracy and speed indicators of TNT series models are shown in the following table. For more introduction, please refer to: [TNT series model documents](../models/TNT_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| TNT_small | 0.8121 |0.9563 | | | 4.83 | 23.68 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/TNT_small_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/TNT_small_infer.tar) | + +**Note**: Both `mean` and `std` in the data preprocessing part of the TNT model `NormalizeImage` are 0.5. + + + +## 23. Other models + +The accuracy and speed indicators of AlexNet [[18](#ref18)], SqueezeNet series [[19](#ref19)], VGG series [[20](#ref20)], DarkNet53 [[21](#ref21)] and other models are shown in the following table. For more information, please refer to: [Other model documents](../models/Others_en.md). + +| Model | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | Pretrained Model Download Address | Inference Model Download Address | +|------------------------|-----------|-----------|------------------|------------------|----------|-----------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| +| AlexNet | 0.567 | 0.792 | 0.81 | 1.50 | 2.33 | 0.71 | 61.10 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/AlexNet_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/AlexNet_infer.tar) | +| SqueezeNet1_0 | 0.596 | 0.817 | 0.68 | 1.64 | 2.62 | 0.78 | 1.25 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SqueezeNet1_0_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SqueezeNet1_0_infer.tar) | +| SqueezeNet1_1 | 0.601 | 0.819 | 0.62 | 1.30 | 2.09 | 0.35 | 1.24 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SqueezeNet1_1_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SqueezeNet1_1_infer.tar) | +| VGG11 | 0.693 | 0.891 | 1.72 | 4.15 | 7.24 | 7.61 | 132.86 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG11_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG11_infer.tar) | +| VGG13 | 0.700 | 0.894 | 2.02 | 5.28 | 9.54 | 11.31 | 133.05 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG13_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG13_infer.tar) | +| VGG16 | 0.720 | 0.907 | 2.48 | 6.79 | 12.33 | 15.470 | 138.35 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG16_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG16_infer.tar) | +| VGG19 | 0.726 | 0.909 | 2.93 | 8.28 | 15.21 | 19.63 | 143.66 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG19_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG19_infer.tar) | +| DarkNet53 | 0.780 | 0.941 | 2.79 | 6.42 | 10.89 | 9.31 | 41.65 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DarkNet53_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DarkNet53_infer.tar) | + + + +## Reference + +[1] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778. + +[2] He T, Zhang Z, Zhang H, et al. Bag of tricks for image classification with convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 558-567. + +[3] Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[C]//Proceedings of the IEEE International Conference on Computer Vision. 2019: 1314-1324. + +[4] Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4510-4520. + +[5] Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017. + +[6] Ma N, Zhang X, Zheng H T, et al. Shufflenet v2: Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 116-131. + +[7] Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1492-1500. + +[8] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7132-7141. + +[9] Gao S, Cheng M M, Zhao K, et al. Res2net: A new multi-scale backbone architecture[J]. IEEE transactions on pattern analysis and machine intelligence, 2019. + +[10] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1-9. + +[11] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-resnet and the impact of residual connections on learning[C]//Thirty-first AAAI conference on artificial intelligence. 2017. + +[12] Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1251-1258. + +[13] Wang J, Sun K, Cheng T, et al. Deep high-resolution representation learning for visual recognition[J]. arXiv preprint arXiv:1908.07919, 2019. + +[14] Chen Y, Li J, Xiao H, et al. Dual path networks[C]//Advances in neural information processing systems. 2017: 4467-4475. + +[15] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708. + +[16] Tan M, Le Q V. Efficientnet: Rethinking model scaling for convolutional neural networks[J]. arXiv preprint arXiv:1905.11946, 2019. + +[17] Mahajan D, Girshick R, Ramanathan V, et al. Exploring the limits of weakly supervised pretraining[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 181-196. + +[18] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105. + +[19] Iandola F N, Han S, Moskewicz M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size[J]. arXiv preprint arXiv:1602.07360, 2016. + +[20] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014. + +[21] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788. + +[22] Ding X, Guo Y, Ding G, et al. Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks[C]//Proceedings of the IEEE International Conference on Computer Vision. 2019: 1911-1920. + +[23] Han K, Wang Y, Tian Q, et al. GhostNet: More features from cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 1580-1589. + +[24] Zhang H, Wu C, Zhang Z, et al. Resnest: Split-attention networks[J]. arXiv preprint arXiv:2004.08955, 2020. + +[25] Radosavovic I, Kosaraju R P, Girshick R, et al. Designing network design spaces[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 10428-10436. + +[26] C.Szegedy, V.Vanhoucke, S.Ioffe, J.Shlens, and Z.Wojna. Rethinking the inception architecture for computer vision. arXiv preprint arXiv:1512.00567, 2015. + +[27] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin and Baining Guo. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. + +[28]Cheng Cui, Tingquan Gao, Shengyu Wei, Yuning Du, Ruoyu Guo, Shuilong Dong, Bin Lu, Ying Zhou, Xueying Lv, Qiwen Liu, Xiaoguang Hu, Dianhai Yu, Yanjun Ma. PP-LCNet: A Lightweight CPU Convolutional Neural Network. + +[29]Mingxing Tan, Quoc V. Le. MixConv: Mixed Depthwise Convolutional Kernels. + +[30]Dongyoon Han, Sangdoo Yun, Byeongho Heo, YoungJoon Yoo. Rethinking Channel Dimensions for Efficient Model Design. + +[31]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby. AN IMAGE IS WORTH 16X16 WORDS: +TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE. + +[32]Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, Herve Jegou. Training data-efficient image transformers & distillation through attention. + +[33]Benjamin Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Herve Jegou, Matthijs Douze. LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference. + +[34]Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, Chunhua Shen. Twins: Revisiting the Design of Spatial Attention in Vision Transformers. + +[35]Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, Yunhe Wang. Transformer in Transformer. + +[36]Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, Jian Sun. RepVGG: Making VGG-style ConvNets Great Again. + +[37]Ping Chao, Chao-Yang Kao, Yu-Shan Ruan, Chien-Hsiang Huang, Youn-Long Lin. HarDNet: A Low Memory Traffic Network. + +[38]Fisher Yu, Dequan Wang, Evan Shelhamer, Trevor Darrell. Deep Layer Aggregation. + +[39]Duo Lim Jie Hu, Changhu Wang, Xiangtai Li, Qi She, Lei Zhu, Tong Zhang, Qifeng Chen. Involution: Inverting the Inherence of Convolution for Visual Recognition. diff --git a/src/PaddleClas/docs/en/algorithm_introduction/image_classification_en.md b/src/PaddleClas/docs/en/algorithm_introduction/image_classification_en.md new file mode 100644 index 0000000..fa2319c --- /dev/null +++ b/src/PaddleClas/docs/en/algorithm_introduction/image_classification_en.md @@ -0,0 +1,78 @@ +# Image Classification Task Introduction +## Catalogue + +- [1. Dataset Introduction](#1) + - [1.1 ImageNet-1k](#1.1) + - [1.2 CIFAR-10/CIFAR-100](#1.2) +- [2. Image Classification Process](#2) + - [2.1 Data and its Preprocessing](#2.1) + - [2.2 Prepare the model](#2.2) + - [2.3 Train the model](#2.3) + - [2.4 Evaluate the model](#2.4) +- [3. Main Algorithms Introduction](#3) + +Image Classification is a fundamental task that classifies the image by semantic information and assigns it to a specific label. Image Classification is the foundation of Computer Vision tasks, such as object detection, image segmentation, object tracking, and behavior analysis. Image Classification enjoys comprehensive applications, including face recognition and smart video analysis in the security and protection field, traffic scenario recognition in the traffic field, image retrieval and electronic photo album classification in the internet industry, and image recognition in the medical industry. + +Generally speaking, Image Classification attempts to fully describe the whole image by feature engineering and assigns labels by a classifier. Hence, how to extract the features of images is the essential part. Before we have deep learning, the most adopted classification method is the Bag of Words model. However, Image Classification based on deep learning can learn the hierarchical feature description by supervised and unsupervised learning, replacing the manual image feature selection. Recently, Convolution Neural Network (CNN) in deep learning gives an awesome performance in the image field. It uses pixel information as the input to get all the information to the maximum extent. Additionally, since the model uses convolution to extract features, the classification result is the output. Thus, this end-to-end method performs well and is widespread. + +Image Classification is a basic but important field in computer vision, whose research results have a lasting impact on the development of computer vision and even deep learning. Image classification has many sub-fields, such as multi-label image classification and fine-grained image classification. Here we only brief on the single-label image classification. + + + +## 1. Dataset Introduction + + + +### 1.1 ImageNet-1k + +The ImageNet project is a large-scale visual database for the research of visual object recognition software. More than 14 million images have been annotated manually to point out objects in the picture, and at least 1 million images are provided with borders. ImageNet-1k is a subset of the ImageNet dataset, which contains 1000 categories. The training set contains 1281167 image data, and the validation set contains 50,000 image data. Since 2010, ImageNet began to hold an annual image classification competition, namely, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) with ImageNet-1k as its specified dataset. To date, ImageNet-1k has become one of the most significant contributors to the development of computer vision, based on which numerous initial models of downstream computer vision tasks are trained. + + + +### 1.2 CIFAR-10/CIFAR-100 + +The CIFAR-10 data set consists of 60,000 color images of 10 categories with an image resolution of 32x32, and each category has 6000 images, including 5000 in the training set and 1000 in the validation set. The 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. The CIFAR-100 dataset is an extension of CIFAR-10 and consists of 60,000 color images of 100 classes with an image resolution of 32x32, and each class has 600 images, including 500 in the training set and 100 in the validation set. Researchers can try different algorithms quickly due to their small scale. These two data sets are also commonly used for testing the quality of models in image classification. + + + +## 2. Image Classification Process + +The prepared training data is correspondingly preprocessed and then passed through the image classification model. The output of the model and the real label are used in a cross-entropy loss function which describes the convergence direction of the model. An image classification model can be obtained by repeatedly traversing all the image data input models, conducting the corresponding gradient descent for the final loss function through some optimizers, returning the gradient information to the model, and updating the weight of the model. + + + +### 2.1 Data and its Preprocessing + +The quality and quantity of data often determine the performance of a model. In the field of image classification, data includes images and labels. In most cases, labeled data is scarce to an extent that hard to saturate the model. In order to enable the model to learn more image features, a lot of image transformation or data augmentation is required before the image enters the model, so as to ensure the diversity of input data, hence better generalization capabilities of the model. PaddleClas provides standard image transformation for training ImageNet-1k and 8 data augmentation methods. For related codes, please refer to [Data Preprocess](https://github.com/PaddlePaddle/PaddleClas/blob/develop/ppcls/data/preprocess)，and the configuration file to [Data Augmentation Configuration File](https://github.com/PaddlePaddle/PaddleClas/blob/develop/ppcls/configs/ImageNet/DataAugment). + + + +### 2.2 Prepare the Model + +After the data is settled, the model often determines the upper limit of the final accuracy. In the field of image classification, classic models emerge in endlessly. PaddleClas provides 36 series, or a total of 164 ImageNet pre-trained models. For specific accuracy, speed and other indicators, please refer to [Backbone Network Introduction](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/en/models). + + + +### 2.3 Train the Model + +After preparing the data and model, you can start training the model and updating the parameters of the model. After many iterations, a trained model can finally be obtained for image classification tasks. The training process of image classification requires a lot of experience and involves the setting of many hyperparameters. PaddleClas provides a series of [training tuning methods](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/en/models/Tricks_en.md), which can help you quickly obtain a high-precision model. + + + +### 2.4 Evaluate the Model + +After a model is trained, the evaluation results of the model on the validation set can determine the performance of the model. The evaluation index is generally Top1-Acc or Top5-Acc, and the higher the index, the better the model performance. + + + +## 3. Main Algorithms Introduction + +- LeNet: Yan LeCun et al. first applied convolutional neural networks to image classification tasks in the 1990s, and creatively proposed LeNet, which achieved great success in handwritten digit recognition tasks. +- AlexNet: Alex Krizhevsky et al. proposed AlexNet in 2012 and applied it to ImageNet, and won the 2012 ImageNet classification competition. Since then, a deep learning boom is created. +- VGG: Simonyan and Zisserman put forward the VGG network structure in 2014. This network structure uses a smaller convolution kernel to stack the entire network, achieving better performance in ImageNet classification and providing new ideas for the subsequent network structure design. +- GoogLeNet: Christian Szegedy et al. presented GoogLeNet in 2014. This network uses a multi-branch structure and a global average pooling layer (GAP). While maintaining the accuracy of the model, the amount of model storage and calculation witnesses a drastic decrease. The network won the 2014 ImageNet classification competition. +- ResNet: Kaiming He et al. delivered ResNet in 2015, which deepened the depth of the network by introducing a residual module, reducing the recognition error rate of ImageNet classification to 3.6%, which exceeded the recognition accuracy of normal human eyes for the first time. +- DenseNet: Huang Gao et al. proposed DenseNet in 2017. The network designed a denser connected block and achieved higher performance with a smaller amount of parameters. +- EfficientNet: Mingxing Tan et al. introduced EfficientNet in 2019. This network balances the width of the network, the depth of the network, and the resolution of the input image. With the same FLOPS and parameters, it reaches the state-of-the-art results. + +For more algorithm introduction, please refer to [Algorithm Introduction](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/en/models). diff --git a/src/PaddleClas/docs/en/algorithm_introduction/index.rst b/src/PaddleClas/docs/en/algorithm_introduction/index.rst new file mode 100644 index 0000000..e0fad79 --- /dev/null +++ b/src/PaddleClas/docs/en/algorithm_introduction/index.rst @@ -0,0 +1,12 @@ +algorithm_introduction +================================ + +.. toctree:: + :maxdepth: 2 + + image_classification_en.md + metric_learning_en.md + knowledge_distillation_en.md + model_prune_quantization_en.md + ImageNet_models_en.md + DataAugmentation_en.md diff --git a/src/PaddleClas/docs/en/algorithm_introduction/knowledge_distillation_en.md b/src/PaddleClas/docs/en/algorithm_introduction/knowledge_distillation_en.md new file mode 100644 index 0000000..f528569 --- /dev/null +++ b/src/PaddleClas/docs/en/algorithm_introduction/knowledge_distillation_en.md @@ -0,0 +1,94 @@ +# Knowledge Distillation + +--- +## Content + +* [1. Introduction of model compression methods](#1) +* [2. Application of knowledge distillation](#2) +* [3. Overview of knowledge distillation methods](#3) + * [3.1 Response based distillation](#3.1) + * [3.2 Feature based distillation](#3.2) + * [3.3 Relation based distillation](#3.3) +* [4. Reference](#4) + + + +## 1. Introduction of model compression methods + +In recent years, deep neural networks have been proved to be an extremely effective method for solving problems in the fields of computer vision and natural language processing. A suitable neural network architecture might performs better than traditional algorithms mostly. + +When the amount of data is large enough, increasing the model parameters with a reasonable method can significantly improve the model performance, but this brings about the problem of a sharp increase of the model complexity. It costs more for larger models. + +Parameter redundancy exists in deep neural networks generally. At present, there are several mainstream methods to compress the model and reduce parameters. Such as pruning, quantization, knowledge distillation, etc. Knowledge distillation refers to the use of a teacher model to guide the student model to learn specific tasks to ensure that the small model obtains relatively large performance, and even has comparable performance with the large model [1]. + + +Currently, knowledge distillation methods can be roughly divided into the following three types. + +* Response based distillation: Output of student model is guided by the teacher model for +* Feature based distillation: Inner feature map of student model is guided by the teacher model. +* Relation based distillation: For different samples, the teacher model and the student model are used to calculate the correlation of the feature map between the samples, the final goal is to make sure that correlation matrix of student model and the teacher model are as consistent as possible. + + + + +## 2. Application of knowledge distillation + +Knowledge distillation algorithm is widely used in lightweight tasks. For tasks that need to meet specific accuracy, by using the knowledge distillation method, we can achieve the required accuracy with a smaller model, thereby reducing model deployment cost. + + +What's more, for the same model structure, pre-trained models obtained by knowledge distillation often performs better, and these pre-trained models can also improve performance of the downstream tasks. For example, a pre-trained image classification model with higher accuracy can also help other tasks obtain significant accuracy gains such as target detection, image segmentation, OCR, and video classification. + + + +## 3. Overview of knowledge distillation methods + + + +### 3.1 Response based distillation + + +Knowledge distillation algorithm is firstly proposed by Hinton, which is called KD. In addition to base cross entropy loss, KL divergence loss between output of student model and teacher model is also added into the total training loss. It's noted that a larger teacher model is needed to guide the training process of the student model. + +PaddleClas proposed a simple but useful knowledge distillation algorithm canlled SSLD [6], Labels are not needed for SSLD, so unlabeled data can also be used for training. Accuracy of 15 models has more than 3% improvement using SSLD. + +Teacher model is needed for the above-mentioned distillation method to guide the student model training process. Deep Mutual Learning (DML) is then proposed [7], for which two models with same architecture learn from each other to obtain higher accuracy. Compared with KD and other knowledge distillation algorithms that rely on large teacher models, DML is free of dependence on large teacher models. The distillation training process is simpler. + + + +### 3.2 Feature based distillation + +Heo et al. proposed OverHaul [8], which calculates the feature map distance between the student model and the teacher model, as distillation loss. Here, feature map alignment of the student model and the teacher model is used to ensure that the feature maps' distance can be calculated. + +Feature based distillation can also be integrated with the response based knowledge distillation algorithm in Chapter 3.1, which means both the inner feature map and output of the student model are guided during the training process. For the DML method, this integration process is simpler, because the alignment process is not needed since the two models' architectures are absolutely same. This integration process is used in the PP-OCRv2 system, which ultimately greatly improves the accuracy of the OCR text recognition model. + + + +### 3.3 Relation based distillation + +The papers in chapters `3.1` and `3.2` mainly consider the inner feature map or final output of the student model and the teacher model. These knowledge distillation algorithms only focus on the output for single sample, but do not consider the output relationship between different samples. + +Park et al. proposed RKD [10], a relationship-based knowledge distillation algorithm. In RKD, the relationship between different samples is further considered, and two loss functions are used, which are the second-order distance loss (distance-wise) and the third-order angle loss (angle-wise). For the final distillation loss, KD loss and RKD loss are considered at the same time. The final accuracy is better than the accuracy of the model obtained just using KD loss. + + + +## 4. Reference + +[1] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015. + +[2] Bagherinezhad H, Horton M, Rastegari M, et al. Label refinery: Improving imagenet classification through label progression[J]. arXiv preprint arXiv:1805.02641, 2018. + +[3] Yalniz I Z, Jégou H, Chen K, et al. Billion-scale semi-supervised learning for image classification[J]. arXiv preprint arXiv:1905.00546, 2019. + +[4] Cubuk E D, Zoph B, Mane D, et al. Autoaugment: Learning augmentation strategies from data[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2019: 113-123. + +[5] Touvron H, Vedaldi A, Douze M, et al. Fixing the train-test resolution discrepancy[C]//Advances in Neural Information Processing Systems. 2019: 8250-8260. + +[6] Cui C, Guo R, Du Y, et al. Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones[J]. arXiv preprint arXiv:2103.05959, 2021. + +[7] Zhang Y, Xiang T, Hospedales T M, et al. Deep mutual learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4320-4328. + +[8] Heo B, Kim J, Yun S, et al. A comprehensive overhaul of feature distillation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 1921-1930. + +[9] Du Y, Li C, Guo R, et al. PP-OCRv2: Bag of Tricks for Ultra Lightweight OCR System[J]. arXiv preprint arXiv:2109.03144, 2021. + +[10] Park W, Kim D, Lu Y, et al. Relational knowledge distillation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 3967-3976. diff --git a/src/PaddleClas/docs/en/algorithm_introduction/metric_learning_en.md b/src/PaddleClas/docs/en/algorithm_introduction/metric_learning_en.md new file mode 100644 index 0000000..e8e4096 --- /dev/null +++ b/src/PaddleClas/docs/en/algorithm_introduction/metric_learning_en.md @@ -0,0 +1,40 @@ +# Metric Learning + +## Catalogue + +- [1.Introduction](#1) +- [2.Applications](#2) +- [3.Algorithms](#3) + - [3.1 Classification based](#3.1) + - [3.2 Pairwise based](#3.2) + + +## 1.Introduction + +Measuring the distance between data is a common practice in machine learning. Generally speaking, Euclidean Distance, Inner Product, or Cosine Similarity are all available to calculate measurable data. However, the same operation can hardly be replicated on unstructured data, such as calculating the compatibility between a video and a piece of music. Despite the difficulty in performing the aforementioned vector operation directly due to varied data formats, priori knowledge tells that ED(laugh_video, laugh_music) < ED(laugh_video, blue_music). And how to effectively characterize this "distance"? This is exactly the focus of Metric Learning. + +Metric learning, known as Distance Metric Learning, is to automatically construct a task-specific metric function based on training data in the form of machine learning. As shown in the figure below, the goal of Metric learning is to learn a transformation function (either linear or nonlinear) L that maps data points from the original vector space to a new one in which similar points are closer together and non-similar points are further apart, making the metric more task-appropriate. And Deep Metric Learning fits the transformation function by adopting a deep neural network. ![example](../../images/ml_illustration.jpg) + + +## 2.Applications + +Metric Learning technologies are widely applied in real life, such as Face Recognition, Person ReID, Image Retrieval, Fine-grained classification, etc. With the growing prevalence of deep learning in industrial practice, Deep Metric Learning (DML) emerges as the current research direction. + +Normally, DML consists of three parts: a feature extraction network for map embedding, a sampling strategy to combine samples in a mini-batch into multiple sub-sets, and a loss function to compute the loss on each sub-set. Please refer to the figure below: ![image](../../images/ml_pipeline.jpg) + + +## 3.Algorithms + +Two learning paradigms are adopted in Metric Learning: + + +### 3.1 Classification based: + +This refers to methods based on classification labels. They learn the effective feature representation by classifying each sample into the correct category and require the participation of the explicit labels of each sample in the Loss calculation during the learning process. Common algorithms include [L2-Softmax](https://arxiv.org/abs/1703.09507), [Large-margin Softmax](https://arxiv.org/abs/1612.02295), [Angular Softmax](https://arxiv.org/pdf/1704.08063.pdf), [NormFace](https://arxiv.org/abs/1704.06369), [AM-Softmax](https://arxiv.org/abs/1801.05599), [CosFace](https://arxiv.org/abs/1801.09414), [ArcFace](https://arxiv.org/abs/1801.07698), etc. These methods are also called proxy-based, because what they optimize is essentially the similarity between a sample and a set of proxies. + + +### 3.2 Pairwise based: + +This refers to the learning paradigm based on paired samples. It takes sample pairs as input and obtains an effective feature representation by directly learning the similarity between these pairs. Common algorithms include [Contrastive loss](http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf), [ Triplet loss](https://arxiv.org/abs/1503.03832), [Lifted-Structure loss](https://arxiv.org/abs/1511.06452), [N-pair loss](https://), [Multi-Similarity loss](https://arxiv.org/pdf/1904.06627.pdf), etc. + +[CircleLoss](https://arxiv.org/abs/2002.10857), released in 2020, unifies the two learning paradigms from a fresh perspective, prompting researchers and practitioners' further reflection on Metric Learning. diff --git a/src/PaddleClas/docs/en/algorithm_introduction/model_prune_quantization_en.md b/src/PaddleClas/docs/en/algorithm_introduction/model_prune_quantization_en.md new file mode 100644 index 0000000..14568a3 --- /dev/null +++ b/src/PaddleClas/docs/en/algorithm_introduction/model_prune_quantization_en.md @@ -0,0 +1,52 @@ +# Algorithms of Model Pruning and Quantization + +Deep learning limits the deployment of corresponding models in some scenarios and devices due to its computational complexity or parameter redundancy, thus requiring model compression, optimization acceleration. Model compression algorithms can effectively decrease parameter redundancy, thus reducing storage footprint, communication bandwidth, and computational complexity, which are conducive to the application and deployment of models of deep learning. Among them, model quantization and pruning enjoy great popularity. In PaddleClas, the following two algorithms should be mainly applied. + +- Quantization: PACT +- Pruning: FPGM + +See [PaddeSlim](https://github.com/PaddlePaddle/PaddleSlim/) for detailed parameters. + +## Catlogue + +- [1. PACT](#1) +- [2. FPGM](#2) + + + +## 1. PACT + +The model quantization comprises two main parts, the quantization of the Weight and the Activation. Simultaneous quantization of the two parts is necessary to maximize the computational efficiency gain. The weight can be distributed as compactly as possible by means of network regularization to reduce outliers and uneven distribution, while there is a lack of effective means for activation. + +**PACT (PArameterized Clipping acTivation)** is a new quantization method that minimizes the loss of accuracy, or even achieves great accuracy by removing some outliers before the quantization of activation. The method was proposed when the author found that "the quantized activation differed significantly from the full accuracy results when the weight quantization is adopted". The author also found that the quantization of activation can cause a great error (as a result of RELU, the range of activation is infinite compared to the weight which is basically within 0 to 1), so the activation function **clipped RELU** was introduced. The clipping ceiling, i.e., $α$, is a learnable parameter, which ensures that each layer can learn a different quantization range through training and minimizes the rounding error caused by quantization. The schematic diagram of quantization is shown below. **PACT** solves the problem by continuously trimming the activation range so that the activation distribution is narrowed, thus reducing the quantization mapping loss. It can acquire a more reasonable quantization scale and cut the quantization loss by clipping the activation, thus reducing the outliers in the activation distribution. + +![img](../../images/algorithm_introduction/quantization.jpg) + +The quantization formula of **PACT** is as follows: + +![img](../../images/algorithm_introduction/quantization_formula.png) + +It is shown that PACT is about adopting the above quantization as a substitute for the *ReLU* function to clip the part greater than zero with a threshold of $a$. However, the above formula is further improved in *PaddleSlim* as follows: + +![img](../../images/algorithm_introduction/quantization_formula_slim.png) + + + +After the above improvement, *PACT* preprocessing is inserted between the activation and the OP (convolution, full connection, etc.) to be quantized, which not only clips the distribution greater than 0 but also perform the same for the part less than 0, so as to better obtain the range to be quantized and minimize the quantization loss. At the same time, the clipping threshold is a trainable parameter, which can be detected automatically and reasonably by the model during the quantization training, thus further lowering the quantization accuracy loss. + +For specific algorithm parameters, please refer to [Introduction to Parameters](https://github.com/PaddlePaddle/PaddleSlim/blob/release/2.0.0/docs/zh_cn/api_cn/dygraph/quanter/qat.rst#qat) in PaddleSlim. + + + +## 2. FPGM + +Model pruning is an essential practice to reduce the model size and improve inference efficiency. In previous articles on network pruning, the norm of the network filter is generally adopted to measure its importance, **the smaller the norm value, the less important the filter is** and the more significant it will be to clip it from the network. **FPGM** believes that the previous approach relies on the following two points: + +- The deviation of the filter's norm should be large so that important and unimportant filters can be well separated +- The norm of the unimportant filter should be small enough + +Based on this, **FPGM** takes advantage of the geometric center property of the filter. Since filters near the center can be expressed by others, they can be eliminated, thus avoiding the above two pruning conditions. As a result, the pruning is conducted in consideration of the redundancy of information instead of a small norm. The following figure shows how the **FPGM** differs from the previous method, see [paper](https://openaccess.thecvf.com/content_CVPR_2019/papers/He_Filter_Pruning_via_Geometric_Median_) for more details. + +![img](../../images/algorithm_introduction/fpgm.png) + +For specific algorithm parameters, please refer to [Introduction to Parameters](https://github.com/PaddlePaddle/PaddleSlim/blob/release/2.0.0/docs/zh_cn/api_cn/dygraph/pruners/fpgm_filter_pruner.rst#fpgmfilterpruner) in PaddleSlim. diff --git a/src/PaddleClas/docs/en/change_log.md b/src/PaddleClas/docs/en/change_log.md new file mode 100644 index 0000000..406a49e --- /dev/null +++ b/src/PaddleClas/docs/en/change_log.md @@ -0,0 +1,3 @@ +# Release Notes + +* 2020.04.14: first commit diff --git a/src/PaddleClas/docs/en/conf.py b/src/PaddleClas/docs/en/conf.py new file mode 100644 index 0000000..fef10ee --- /dev/null +++ b/src/PaddleClas/docs/en/conf.py @@ -0,0 +1,65 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) +import sphinx_rtd_theme +from recommonmark.parser import CommonMarkParser +# -- Project information ----------------------------------------------------- + +project = 'PaddleClas-en' +copyright = '2022, PaddleClas' +author = 'PaddleClas' + +# The full version, including alpha/beta/rc tags +release = '2.3' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +source_parsers = { + '.md': CommonMarkParser, +} +source_suffix = ['.rst', '.md'] +extensions = [ + 'recommonmark', + 'sphinx_markdown_tables' + ] +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The root document. +root_doc = 'doc_en' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +# 更改文档配色 +html_theme = "sphinx_rtd_theme" +html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] diff --git a/src/PaddleClas/docs/en/data_preparation/classification_dataset_en.md b/src/PaddleClas/docs/en/data_preparation/classification_dataset_en.md new file mode 100644 index 0000000..3903a00 --- /dev/null +++ b/src/PaddleClas/docs/en/data_preparation/classification_dataset_en.md @@ -0,0 +1,130 @@ +# Image Classification Datasets + +This document elaborates on the dataset format adopted by PaddleClas for image classification tasks, as well as other common datasets in this field. + +------ + +## Catalogue + +- [1.Dataset Format](#1) +- [2.Common Datasets for Image Classification](#2) + - [2.1 ImageNet1k](#2.1) + - [2.2 Flowers102](#2.2) + - [2.3 CIFAR10 / CIFAR100](#2.3) + - [2.4 MNIST](#2.4) + - [2.5 NUS-WIDE](#2.5) + + + +## 1.Dataset Format + +PaddleClas adopts `txt` files to assign the training and test sets. Taking the `ImageNet1k` dataset as an example, where `train_list.txt` and `val_list.txt` have the following formats: + +``` +# Separate the image path and annotation with "space" for each line + +# train_list.txt has the following format +train/n01440764/n01440764_10026.JPEG 0 +... + +# val_list.txt has the following format +val/ILSVRC2012_val_00000001.JPEG 65 +... +``` + + + +## 2.Common Datasets for Image Classification + +Here we present a compilation of commonly used image classification datasets, which is continuously updated and expects your supplement. + + +### 2.1 ImageNet1k + +[ImageNet](https://image-net.org/) is a large visual database for visual target recognition research with over 14 million manually labeled images. ImageNet-1k is a subset of the ImageNet dataset, which contains 1000 categories with 1281167 images for the training set and 50000 for the validation set. Since 2010, ImageNet began to hold an annual image classification competition, namely, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) with ImageNet-1k as its specified dataset. To date, ImageNet-1k has become one of the most significant contributors to the development of computer vision, based on which numerous initial models of downstream computer vision tasks are trained. + +| Dataset | Size of Training Set | Size of Test Set | Number of Category | Note | +| ------------------------------------------------------------ | -------------------- | ---------------- | ------------------ | ---- | +| [ImageNet1k](http://www.image-net.org/challenges/LSVRC/2012/) | 1.2M | 50k | 1000 | | + +After downloading the data from official sources, organize it in the following format to train with the ImageNet1k dataset in PaddleClas. + +``` +PaddleClas/dataset/ILSVRC2012/ +|_ train/ +| |_ n01440764 +| | |_ n01440764_10026.JPEG +| | |_ ... +| |_ ... +| | +| |_ n15075141 +| |_ ... +| |_ n15075141_9993.JPEG +|_ val/ +| |_ ILSVRC2012_val_00000001.JPEG +| |_ ... +| |_ ILSVRC2012_val_00050000.JPEG +|_ train_list.txt +|_ val_list.txt +``` + + + +### 2.2 Flowers102 + +| Dataset | Size of Training Set | Size of Test Set | Number of Category | Note | +| ------------------------------------------------------------ | -------------------- | ---------------- | ------------------ | ---- | +| [flowers102](https://www.robots.ox.ac.uk/~vgg/data/flowers/102/) | 1k | 6k | 102 | | + +Unzip the downloaded data to see the following directory. + +``` +jpg/ +setid.mat +imagelabels.mat +``` + +Place the files above under `PaddleClas/dataset/flowers102/` . + +Run `generate_flowers102_list.py` to generate `train_list.txt` and `val_list.txt`: + +``` +python generate_flowers102_list.py jpg train > train_list.txt +python generate_flowers102_list.py jpg valid > val_list.txt +``` + +Structure the data as follows： + +``` +PaddleClas/dataset/flowers102/ +|_ jpg/ +| |_ image_03601.jpg +| |_ ... +| |_ image_02355.jpg +|_ train_list.txt +|_ val_list.txt +``` + + + +### 2.3 CIFAR10 / CIFAR100 + +The CIFAR-10 dataset comprises 60,000 color images of 10 classes with 32x32 image resolution, each with 6,000 images including 5,000 images in the training set and 1,000 images in the validation set. The 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. The CIFAR-100 dataset is an extension of CIFAR-10 and consists of 60,000 color images of 100 classes with 32x32 image resolution, each with 600 images including 500 images in the training set and 100 images in the validation set. + +Website：http://www.cs.toronto.edu/~kriz/cifar.html + + + +### 2.4 MNIST + +MMNIST is a renowned dataset for handwritten digit recognition and is used as an introductory sample for deep learning in many sources. It contains 60,000 images, 50,000 for the training set and 10,000 for the validation set, with a size of 28 * 28. + +Website：http://yann.lecun.com/exdb/mnist/ + + + +### 2.5 NUS-WIDE + +NUS-WIDE is a multi-category dataset. It contains 269,648 images and 81 categories with each image being labeled as one or more of the 81 categories. + +Website：https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html diff --git a/src/PaddleClas/docs/en/data_preparation/index.rst b/src/PaddleClas/docs/en/data_preparation/index.rst new file mode 100644 index 0000000..e668126 --- /dev/null +++ b/src/PaddleClas/docs/en/data_preparation/index.rst @@ -0,0 +1,8 @@ +data_preparation +================================ + +.. toctree:: + :maxdepth: 2 + + recognition_dataset_en.md + classification_dataset_en.md diff --git a/src/PaddleClas/docs/en/data_preparation/recognition_dataset_en.md b/src/PaddleClas/docs/en/data_preparation/recognition_dataset_en.md new file mode 100644 index 0000000..32a2798 --- /dev/null +++ b/src/PaddleClas/docs/en/data_preparation/recognition_dataset_en.md @@ -0,0 +1,141 @@ +# Image Recognition Datasets + +This document elaborates on the dataset format adopted by PaddleClas for image recognition tasks, as well as other common datasets in this field. + +------ + +## Catalogue + +- [1.Dataset Format](#1) +- [2.Common Datasets for Image Recognition](#2) + - [2.1 General Datasets](#2.1) + - [2.2 Vertical Class Datasets](#2.2) + - [2.2.1 Animation Character Recognition](#2.2.1) + - [2.2.2 Product Recognition](#2.2.2) + - [2.2.3 Logo Recognition](#2.2.3) + - [2.2.4 Vehicle Recognition](#2.2.4) + + + +## 1.Dataset Format + +The dataset for the vector search, unlike those for classification tasks, is divided into the following three parts: + +- Train dataset: Used to train the model to learn the image features involved. +- Gallery dataset: Used to provide the gallery data in the vector search task. It can either be the same as the train or query datasets or different, and when it is the same as the train dataset, the category system of the query dataset and train dataset should be the same. +- Query dataset: Used to test the performance of the model. It usually extracts features from each query image of the dataset, followed by distance matching with those in the gallery dataset to get the recognition results, based on which the metrics of the whole query dataset are calculated. + +The above three datasets all adopt `txt` files for assignment. Taking the `CUB_200_2011` dataset as an example, the `train_list.txt` of the train dataset has the following format： + +``` +# Use "space" as the separator +... +train/99/Ovenbird_0136_92859.jpg 99 2 +... +train/99/Ovenbird_0128_93366.jpg 99 6 +... +``` + +The `test_list.txt` of the query dataset (both gallery dataset and query dataset in`CUB_200_2011`) has the following format： + +``` +# Use "space" as the separator +... +test/200/Common_Yellowthroat_0126_190407.jpg 200 1 +... +test/200/Common_Yellowthroat_0114_190501.jpg 200 6 +... +``` + +Each row of data is separated by "space", and the three columns of data stand for the path, label information, and unique id of training data. + +**Note**： + +1. When the gallery dataset and query dataset are the same, to remove the first retrieved data (the images themselves require no evaluation), each data should have its unique id (ensuring that each image has a different id, which can be represented by the row number) for subsequent evaluation of mAP, recall@1, and other metrics. The dataset of yaml configuration file is `VeriWild`. + +2. When the gallery dataset and query dataset are different, there is no need to add a unique id. Both `query_list.txt` and `gallery_list.txt` contain two columns, which are the path and label information of the training data. The dataset of yaml configuration file is ` ImageNetDataset`. + + + +## 2.Common Datasets for Image Recognition + +Here we present a compilation of commonly used image recognition datasets, which is continuously updated and expects your supplement. + + +### 2.1 General Datasets + +- SOP: The SOP dataset is a common product dataset in general recognition research and MetricLearning technology research, which contains 120,053 images of 22,634 products downloaded from eBay.com. There are 59,551 images of 11,318 in the training set and 60,502 images of 11,316 categories in the validation set. + + Website: https://cvgl.stanford.edu/projects/lifted_struct/ + +- Cars196: The Cars dataset contains 16,185 images of 196 categories of cars. The data is classified into 8144 training images and 8041 query images, with each category split roughly in a 50-50 ratio. The classification is normally based on the manufacturing, model and year of the car, e.g. 2012 Tesla Model S or 2012 BMW M3 coupe. + + Website: https://ai.stanford.edu/~jkrause/cars/car_dataset.html + +- CUB_200_2011: The CUB_200_2011 dataset is a fine-grained dataset proposed by the California Institute of Technology (Caltech) in 2010 and is currently the benchmark image dataset for fine-grained classification recognition research. There are 11788 bird images in this dataset with 200 subclasses, including 5994 images in the train dataset and 5794 images in the query dataset. Each image provides label information, the bounding box of the bird, the key part information of the bird, and the attribute of the bird. The dataset is shown in the figure below. + +- In-shop Clothes: In-shop Clothes is one of the 4 subsets of the DeepFashion dataset. It is a seller show image dataset with multi-angle images of each product id being collected in the same folder. The dataset contains 7982 items with 52712 images, each with 463 attributes, Bbox, landmarks, and store descriptions. + + Website： http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html + + + +### 2.2 Vertical Class Datasets + + + +#### 2.2.1 Animation Character Recognition + +- iCartoonFace: iCartoonFace, developed by iQiyi (an online video platform), is the world's largest manual labeled detection and recognition dataset for cartoon characters, which contains more than 5013 cartoon characters and 389,678 high-quality live images. Compared with other datasets, it boasts features of large scale, high quality, rich diversity, and challenging difficulty, making it one of the most commonly used datasets to study cartoon character recognition. + +- Website： http://challenge.ai.iqiyi.com/detail?raceId=5def69ace9fcf68aef76a75d + +- Manga109: Manga109 is a dataset released in May 2020 for the study of cartoon character detection and recognition, which contains 21142 images and is officially banned from commercial use. Manga109-s, a subset of this dataset, is available for industrial use, mainly for tasks such as text detection, sketch line-based search, and character image generation. + + Website：http://www.manga109.org/en/ + +- IIT-CFW: The IIF-CFW dataset contains a total of 8928 labeled cartoon portraits of celebrity characters, covering 100 characters with varying numbers of portraits for each. In addition, it also provides 1000 real face photos (10 real portraits for 100 public figures). This dataset can be employed to study both animation character recognition and cross-modal search tasks. + + Website： http://cvit.iiit.ac.in/research/projects/cvit-projects/cartoonfaces + + + +#### 2.2.2 Product Recognition + +- AliProduct: The AliProduct dataset is the largest open source product dataset. As an SKU-level image classification dataset, it contains 50,000 categories and 3 million images, ranking the first in both aspects in the industry. This dataset covers a large number of household goods, food, etc. Due to its lack of manual annotation, the data is messy and unevenly distributed with many similar product images. + + Website: https://retailvisionworkshop.github.io/recognition_challenge_2020/ + +- Product-10k: Products-10k dataset has all its images from Jingdong Mall, covering 10,000 frequently purchased SKUs that are organized into a hierarchy. In total, there are nearly 190,000 images. In the real application scenario, the distribution of image volume is uneven. All images are manually checked/labeled by a team of production experts. + + Website：https://www.kaggle.com/c/products-10k/data?select=train.csv + +- DeepFashion-Inshop: The same as the common datasets In-shop Clothes. + + + +### 2.2.3 Logo Recognition + +- Logo-2K+: Logo-2K+ is a dataset exclusively for logo image recognition, which contains 10 major categories, 2341 minor categories, and 167,140 images. + + Website： https://github.com/msn199959/Logo-2k-plus-Dataset + +- Tsinghua-Tencent 100K: This dataset is a large traffic sign benchmark dataset based on 100,000 Tencent Street View panoramas. 30,000 traffic sign instances included, it provides 100,000 images covering a wide range of illumination, and weather conditions. Each traffic sign in the benchmark test is labeled with the category, bounding box and pixel mask. A total of 222 categories (0 background + 221 traffic signs) are incorporated. + + Website： https://cg.cs.tsinghua.edu.cn/traffic-sign/ + + + +### 2.2.4 Vehicle Recognition + +- CompCars: The images, 136,726 images of the whole car and 27,618 partial ones, are mainly from network and surveillance data. The network data contains 163 vehicle manufacturers and 1,716 vehicle models and includes the bounding box, viewing angle, and 5 attributes (maximum speed, displacement, number of doors, number of seats, and vehicle type). And the surveillance data comprises 50,000 front view images. + + Website： http://mmlab.ie.cuhk.edu.hk/datasets/comp_cars/ + +- BoxCars: The dataset contains a total of 21,250 vehicles, 63,750 images, 27 vehicle manufacturers, and 148 subcategories. All of them are derived from surveillance data. + + Website： https://github.com/JakubSochor/BoxCars + +- PKU-VD Dataset: The dataset contains two large vehicle datasets (VD1 and VD2) that capture images from real-world unrestricted scenes in two cities. VD1 is obtained from high-resolution traffic cameras, while images in VD2 are acquired from surveillance videos. The authors have performed vehicle detection on the raw data to ensure that each image contains only one vehicle. Due to privacy constraints, all the license numbers have been obscured with black overlays. All images are captured from the front view, and diverse attribute annotations are provided for each image in the dataset, including identification numbers, accurate vehicle models, and colors. VD1 originally contained 1097649 images, 1232 vehicle models, and 11 vehicle colors, and remains 846358 images and 141756 vehicles after removing images with multiple vehicles inside and those taken from the rear of the vehicle. VD2 contains 807260 images, 79763 vehicles, 1112 vehicle models, and 11 vehicle colors. + + Website： https://pkuml.org/resources/pku-vds.html diff --git a/src/PaddleClas/docs/en/doc_en.rst b/src/PaddleClas/docs/en/doc_en.rst new file mode 100644 index 0000000..aa25829 --- /dev/null +++ b/src/PaddleClas/docs/en/doc_en.rst @@ -0,0 +1,23 @@ +Welcome to PaddleClas！ +================================ + +.. toctree:: + :maxdepth: 1 + + introduction/index + installation/index + quick_start/index + image_recognition_pipeline/index + data_preparation/index + models_training/index + inference_deployment/index + models/index + algorithm_introduction/index + advanced_tutorials/index + others/index + faq_series/index + + + + + diff --git a/src/PaddleClas/docs/en/faq_series/faq_2020_s1_en.md b/src/PaddleClas/docs/en/faq_series/faq_2020_s1_en.md new file mode 100644 index 0000000..de5a50c --- /dev/null +++ b/src/PaddleClas/docs/en/faq_series/faq_2020_s1_en.md @@ -0,0 +1,390 @@ +# Image Classification FAQ Summary - 2020 Season 1 + +## Catalogue + +- [1. Issue 1](#1)(2020.11.03) +- [2. Issue 2](#2)(2020.11.11) +- [3. Issue 3](#3)(2020.11.18) +- [4. Issue 4](#4)(2020.12.07) +- [5. Issue 5](#5)(2020.12.17) +- [6. Issue 6](#6)(2020.12.30) + + + +## Issue 1 + +### Q1.1: What can PaddleClas be used for? + +**A**：PaddleClas is an image recognition toolset for industry and academia, helping users train better computer vision models and apply them in real scenarios. + +It provides the whole process of model training, evaluation, inference, and deployment based on image classification to facilitate users' efficient learning. Specifically, PaddleClas contains the following features. + +- PaddleClas provides 36 families of classification network structures (ResNet, ResNet_vd, MobileNetV3, Res2Net, HRNet, etc.) and training configurations, 175 pre-trained models, and performance evaluation and inference for free choice and application. +- PaddleClas provides a variety of inference deployment solutions such as TensorRT inference, python inference, c++ inference, Paddle-Lite inference deployment, PaddleServing, PaddleHub, etc., to facilitate inference deployment in multiple environments. +- PaddleClas provides a simple SSLD knowledge distillation scheme, based on which the recognition accuracy of distillation models register a general improvement of more than 3%. +- PaddleClas provides 8 data augmentation algorithms such as AutoAugment, Cutout, Cutmix, etc. with a detailed introduction, code replication, and evaluation of effectiveness in a unified experimental environment. +- PaddleClas supports CPU/GPU-based adoption in Windows/Linux/MacOS environments. + +### Q1.2: What is the ResNet series model? What are they? Why are they so popular on the server side? + +**A**: ResNet takes the lead to introduce the residual structure, and construct the ResNet network by stacking multiple residual structures. Experiments show that the use of residual blocks can effectively improve convergence speed and accuracy. In PaddleClas, ResNet has such structures containing 18, 34, 50, 101, 152, and 200 layers in order. These models, proposed in 2015, has been validated in different application scenarios, such as classification, detection, segmentation, etc., and has long been optimized by the industry and acquired obvious advantages in terms of speed and accuracy, let alone its well support for the inference of TensorRT and FP16. Therefore, it is recommended to adopt the ResNet series model. Considering their large storage footprint, they are often used on the server side. For more information about ResNet models, please refer to the paper [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385). + +### Q1.3: What's the difference between the structure of ResNet_vd, ResNet and ResNet_vc? + +**A**: The structure of ResNet_va to vd is shown in the figure below. ResNet was first proposed as va structure, in the left feature transformation path (Path A) of the downsampling residual module, the first 1x1 convolution is downsampled, which leads to information loss (the kernel size of the convolution is 1, stride is 2, some features in the input feature graph are not involved in the calculation of convolution). In the vb structure, the downsampling step is adjusted from the first 1x1 convolution at the beginning to the 3x3 convolution in the middle, thus avoiding the loss of information, and the default ResNet model in PaddleClas is ResNet_vb. The vc structure turns the initial 7x7 convolution into 3 3x3 convolutions with almost the same computation and storage size and improved accuracy when the perceptual field remains unchanged. The vd structure is a modification of the feature path (Path B) on the right side of the downsampling residual module, replacing the downsampling with average pooling. This collection of improvements (va->vd), with little extra inference time, and combined with appropriate training strategies, such as label smoothing and mixup data augmentation, can improve the accuracy by up to 2.7%. + +![](../../images/faq/ResNet_vabcd_structure.png) + +### Q1.4 How to choose appropriate ResNet models for the actual scenario? + +**A**: + +Among the ResNet series model, the ResNet_vd model is recommended for it has a significant improvement in accuracy with almost constant inference speed compared to other models. When the batch size=4, the variation of inference time, FLOPs, Params and accuracy for different models on T4 GPU are demonstrated in the [ResNet and its vd series models](../models/ResNet_and_vd_en.md). If you want the smallest possible model storage or the fastest inference speed, please use ResNet18_vd model, and if you want to get the highest possible accuracy, we recommend the ResNet152_vd or ResNet200_vd models. For more information about the ResNet series model, please refer to [ResNet and its vd series models](../models/ResNet_and_vd_en.md) + +- Variation of precision-inference speed + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.png) + +- Variation of precision-params + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.params.png) + +- Variation of precision-flops + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.flops.png) + +### Q1.5 Is conv-bn-relu a fixed form in a block of the network? + +**A**: + +Before the advent of batch-norm, the mainstream convolutional neural networks were fixed in the form of conv-relu. At the moment, conv-bn-relu is the fixed form of blocks in most of the convolutional networks, which is a relatively robust design. Besides, the block in DenseNet is adopted in the form of bn-relu-conv, which is the same combination used in ResNet-V2. In MobileNetV2, the middle layer of some blocks adopts conv-bn instead of the relu activation function to avoid information loss. + +### Q1.6 What's the difference between ResNet34 and ResNet50？ + +**A**: + +There are two different kinds of blocks in the ResNet series, basic-block and bottleneck-block, and the ResNet network is constructed by the stacking of such blocks. basic-block is a stack of two 3x3 convolutional kernels with shortcut, while bottleneck-block is a stack of a 1x1 convolutional kernel, 3x3 convolutional kernel, and 1x1 convolutional kernel with shortcut, so there are two layers in the former one and three in the latter. The number of blocks stacked in ResNet34 and ResNet50 is the same, but the types of stacking are basic-block and bottleneck-block, respectively. + +### Q1.7 Do large convolution kernels necessarily lead to positive returns? + +**A**: + +Not really, increasing all the convolutional kernels in the network may not lead to performance improvement or even the opposite. In the paper [MixConv: Mixed Depthwise Convolutional Kernels](https://arxiv.org/abs/1907.09595), it is pointed out that increasing the size of the convolutional kernels within a certain range plays a positive role in the accuracy improvement, but a size beyond may lead to accuracy loss. Therefore, considering the size of the model and the computation, large convolutional kernels are generally abandoned to design the network. Also, there are experiments on large convolution kernels in the article [PP-LCNet](../models/PP-LCNet_en.md). + + + +## Issue 2 + +### Q2.1: How does PaddleClas train its backbone? + +**A**：The process is as follows: + +- First, create a new model structure file under the folder `ppcls/arch/backbone/model_zoo/`, i.e. your own backbone. You can refer to resnet.py for model construction; +- Then add your own backbone class in `ppcls/arch/backbone/__init__.py`; +- Next, configure the yaml file for training, here you can refer to `ppcls/configs/ImageNet/ResNet/ResNet50.yaml`; +- Now you can start the training. + +### Q2.2: How to transfer the existing models and weights to your own classification tasks? + +**A**: The process is as follows: + +- First, a good pre-training model tends to be better transferred, so it is recommended to adopt a pre-training model with higher accuracy, for instance, series of industry-leading pre-training models provided by PaddleClas; +- Second, determine and train hyperparameters based on the size of the dataset to be transferred, which need to be debugged to find a local optimal value. If you have no relevant experience, it is recommended to start with the learning rate, which generally has a smaller dataset adopting a small learning rate, such as 0.001. In addition, the warmup strategy is suggested for the learning rate to avoid the weight damage of the pre-training model resulting from a large learning rate. During the transfer, the learning rate of different layers in the backbone can also be set, and it is often better to gradually reduce the learning rate from the head to the tail of the network. Data augmentation strategies can also be useful for small datasets, and PaddleClas offers 8 powerful data augmentation strategies for higher accuracy. +- After training, the above process can be iterated repeatedly until a local optimal value is found. + +### Q2.3: Is the default parameter under configs in PaddleClas available for all datasets? + +**A**: + +The default parameter of the configuration file under `ppcls/configs/ImageNet/` in PaddleClas is the training parameter of ImageNet-1k, which is not suitable for all datasets, and the specific datasets need to be further debugged on this basis. + +### Q2.4 The resolution varies for different models in PaddleClas, so what is the standard? + +**A**: + +PaddleClas strictly follows the resolution used by the authors of the paper. Since AlexNet in 2012, most of the convolutional neural networks trained on ImageNet have a resolution of 224x224, and Google adjusted the resolution to 299x299 when designing InceptionV3 to fit the network structure, which was the same one for later Xception and InceptionV4. In addition, in EfficeintNet, the authors analyze that different resolutions should be used for networks of different sizes, so does it in this series. In practical scenarios, it is recommended to adopt the default resolution, but networks with deeper layers or larger widths can also try larger ones. + +### Q2.5 There are many ssld models available in PaddleClas, what is the value of their application? + +**A**: + +There are many ssld pre-training models available in PaddleClas, which obtain better pre-training weights by semi-supervised knowledge distillation, so that the accuracy can be improved by replacing the ssld pre-training models with higher accuracy in transfer tasks or downstream vision tasks without replacing the structure files. For example, in PaddleSeg, [HRNet](../models/HRNet_en.md) , with the weight of the ssld pre-training model, achieves much better accuracy than other same models in the industry; In PaddleDetection, [PP- YOLO](https://github.com/PaddlePaddle/PaddleDetection/blob/release/0.4/configs/ppyolo/README_cn.md)with ssld pre-training weights has further improvement in the already high baseline. The transfer of classification with ssld pre-training weights also yields impressive results, and the benefits of knowledge distillation for the transfer of classification task is detailed in [SSLD Distillation Strategy](../advanced_tutorials/knowledge_distillation_en.md) + + + +## Issue 3 + +### Q3.1: What is the improvement of DenseNet model over ResNet? What are the features or application scenarios? + +**A**: + +DenseNet is designed with a more aggressive dense connectivity mechanism compared to ResNet, which further reduces the number of parameters by considering feature reuse and bypass settings, and mitigates the gradient dispersion problem to some extent. What's more, the model is easier to train and equipped with some regularization effect due to the introduction of dense connectivity. DenseNet is a good choice in image classification scenarios where the amount of data is limited. More information about DenseNet and this series can be found in [DenseNet Models](../models/DPN_DenseNet_en.md). + +### Q3.2: What are the improvements of the DPN network over DenseNet? + +**A**： + +The full name of DPN is Dual Path Networks, or Dual Channel Networks. It is a combination of DenseNet and ResNeXt, which demonstrates that DenseNet can extract new features from the previous layers, while ResNeXt is essentially reuse of features already extracted from the previous layers. The authors further analyze and find that ResNeXt has a high reuse rate for features but low redundancy, while DenseNet can create new features but has high redundancy. Combining the advantages of both structures, the DPN network is designed. Finally, the DPN network achieves better results than ResNeXt and DenseNet with the same FLOPS and number of parameters. More introduction and series models of DPN can be found in [DPN Models](../models/DPN_DenseNet_en.md). + +### Q3.3: How to use multiple models for inference fusion? + +**A**: + +When adopting multiple models for inference, it is recommended to first export the pre-training model as an inference model to get rid of the dependence on the network structure definition, you can refer to [model export script](../../../tools/export_model.py) for model exporting, and then see [inference script for the inference model](../../../deploy/python/predict_cls.py), where you need to create multiple predictors according to the number of employed models. + +### Q3.4: How to add your own data augmentation methods in PaddleClas? + +**A**： + +- For single-image augmentation, you can refer to [Single-image based data augmentation script](../../../ppcls/data/preprocess/ops). Learning from the data operator ` ResizeImage ` or `CropImage` to create a new class, and then implement the corresponding augmentation method in `__call__`. +- For a batch image, you can refer to the [batch data-based data augmentation script](../../../ppcls/data/preprocess/batch_ops). Learning from the data operator `MixupOperator` or `CutmixOperator` to create a new class, and then implement the corresponding augmentation method in `__call__`. + +## Q3.5: How to further accelerate the model training? + +**A**： + +- You can adopt auto-mixed precision training, which can gain a significantly faster speed with almost zero precision loss. Take ResNet50 as an example, the configuration file of auto-mixed precision training in PaddleClas can be found at: [ResNet50_fp16.yml](../../../ppcls/configs/ImageNet/ResNet/ResNet50_fp16.yaml). The main step is to add the following lines to the standard configuration file. + +``` +# mixed precision training +AMP: + scale_loss: 128.0 + use_dynamic_loss_scaling: True + use_pure_fp16: &use_pure_fp16 True +``` + +- You can turn on dali to run the data preprocessing method on GPU. When the model is relatively small (reader accounts for a higher percentage of time consumption), an obviously faster speed can be obtained with dali on, which could be employed by adding `-o Global.use_dali=True` during training. You can refer to [dali installation tutorial](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html#nightly-builds) for more details. + + + +## Issue 4 + +### Q4.1: How many types of model files are there in PaddlePaddle? + +**A**: + +- There are two types of model-related files saved in PaddlePaddle. +- One is the files used for *inference deployment*, including files with the suffixes ``pdiparams``, ``model``, where the ``pdiparams` " file stores model parameter information, and "`model`" file stores model network structure information. For inference deployment files, the `paddle.jit.save` and `paddle.jit.load` interfaces are used for saving and loading. +- Another one is used in the *training tuning* process, including files with the suffixes "`pdparams`" and "`pdopt`", where the "`pdparams` " file stores the model parameters information during training, and "`pdopt`" file stores the optimizer information during model training. For training tuning files, `paddle.save` and `paddle.load` interfaces are used for saving and loading. +- The inference deployment file enables you to build the model network structure and load the model parameters for inference, while the training tuning file allows you to load the model parameters and optimizer information for resuming the training process. + +### Q4.2: What are the innovations of HRNet? + +**A**: + +- In the field of image classification, most neural networks are designed with the idea of extracting high-dimensional features of images. Specifically, the input image usually has a high spatial resolution, and through multi-layer convolution and pooling, a feature graph with lower spatial resolution but higher dimension can be gradually obtained and applied in scenarios such as classification. +- However, the authors of *HRNet* believe that the idea of gradually decreasing spatial resolution is not suitable for scenarios such as target detection (classification task of image region-level) and semantic segmentation (classification task of image pixel-level), because a lot of information is lost in this process and the final learned features can hardly represent the information of the original image at the high spatial resolution, while both the task of region-level and pixel-level classification are very sensitive to spatial accuracy. +- Therefore, the authors of *HRNet* propose the idea of paralleling feature graphs with different spatial resolutions, while in contrast, neural networks such as *VGG* cascade feature graphs with different spatial resolutions by different convolutional pooling layers. Moreover, *HRNet* connects feature graphs of equal depth and disparate spatial resolutions, so that the information can be fully exchanged. The specific network structure is shown in the figure below. + +![](../../images/faq/HRNet.png) + +### Q4.3: In HRNet, how are connections made between feature graphs with different spatial resolutions? + +**A**: + +- First, in *HRNet*, the *3 × 3* convolution with *stride* of *2* can be used to obtain a feature graph with low spatial resolution but higher dimension; and for those with low spatial resolution, the *1 × 1* convolution is first used to match the number of channels, and then the nearest neighbor interpolation is used for upsampling to obtain a feature graph with the same spatial resolution and number of channels as the high spatial resolution graph. And for the feature map with the same spatial resolution, the constant mapping can be performed directly. The details are shown in the following figure. + +![](../../images/faq/HRNet_block.png) + +### Q4.4: What does "SE" in the model stand for? + +**A**: + +- SE indicates that the model uses an SE structure, which is derived from the winning solution of the 2017 ImageNet classification competition *Squeeze-and-Excitation Networks (SENet)*, and can be migrated to any other networks. The *scale* vector dimension is the same as the number of channels in the feature map, and the value in each dimension of the learned *scale* vector indicates the enhancement or weakening of the feature channels in that dimension, so as to enhance the important feature channel and weaken the unimportant ones, thus making the extracted features more directional. + +### Q4.5: How does the SE structure implemented? + +![](../../images/faq/SE_structure.png) + +**A**: + +- The *SE* structure is shown in the figure above. First, *Ftr* represents the regular convolution operation, and *X* and *U* are the input and output feature maps of *Ftr*. After obtaining the feature map *U*, operate the *Fsq* and *Fex* to obtain the *scale* vector, which has a dimension of *C*, the same as the number of *U* channels, so it can be applied to *U* by multiplying, and then obtain *X~*. +- Specifically, *Fsq* is the *Global Average Pooling* operation, which is called *Squeeze* by *SENet* authors because it compresses *U* from *C × H × W* to *C × 1 × 1*, and then conduct the *Fex* operation on the output of *Fsq*. +- The *Fex* operation represents two full joins, which is referred to as *Excitation* by the authors. The first full join compresses the dimension of the vector from *1 × 1 × C* to *1 × 1 × C/r*, followed by the use of *RELU*, and then restores the dimension of the vector to *C* by the second full join. The purpose of this operation is to reduce the computation. The *SENet* authors conclude experimentally that a balance between gain and computational effort could be obtained when *r=16*. +- For *Fsq*, the key is to obtain the vector of *C* dimension, so it is not limited to the *Global Average Pooling*. The *SENet* authors believe that the final *scale* is applied to *U* separately by channel, so it is necessary to calculate the corresponding *scale* based on the information of the corresponding channel, so the simplest *Global Average Pooling* is adopted, and the final *scale* vector represents the distribution between different channels, ignoring the situation in the same channel. +- For *Fex*, its role is to find the distribution based on all the training data through training on each *mini batch*. Since our training is performed on *mini batches* and the *scale* based on all training data is the best, we can adopt the *Fex* to train on each *mini batch* to obtain a more reliable *scale*. + + + +## Issue 5 + +### Q5.1 How to choose an optimizer? + +**A**: + +Since the emergence of deep learning, there has been a lot of research on optimizers, which aim to minimize the loss function to find the right weights for a given task. Currently, the main optimizers used in the industry are SGD, RMSProp, Adam, AdaDelt, etc. Among them, since the SGD optimizer with momentum is widely used in academia and industry (only for classification tasks), most of the models we published also adopt this optimizer to achieve gradient descent of the loss function. It has two disadvantages, one is the slow convergence speed, and the other is the reliance on experiences of the initial learning rate setting. However, if the initial learning rate is set properly with the sufficient number of iterations, the optimizer will also stand out among many other optimizers, obtaining higher accuracy on the validation set. Some optimizers with adaptive learning rates, such as Adam and RMSProp, tend to converge fast, but the final convergence accuracy will be slightly worse. If you pursue faster convergence speed, we recommend using these adaptive learning rate optimizers, and SGD optimizers with momentum for higher convergence accuracy. The specific information of the dataset is as follows: + +- ImageNet-1k: It is recommended to use the SGD optimizer with momentum only. +- Other datasets (ImageNet-1k pre-training by default): When loading the pre-training model, you can consider an optimizer such as Adam (which may work better), but the SGD optimizer with momentum is definitely a good solution. + +In addition, to further speed up the training, Lookahead optimizer is also a good choice. On ImageNet-1k, it can guarantee the same convergence accuracy at a faster rate, but the performance is less stable on some datasets and requires further tuning. + +### Q5.2 How to set the initial learning rate and the learning rate decay strategy? + +**A**: The choice of learning rate is often related to the optimizer as well as the data and the task. The learning rate determines how quickly the network weights are updated. The lower the learning rate, the slower the loss function will change. While using a low learning rate ensures that no local minimal values are missed, it also means that it takes longer to converge, especially if trapped in a plateau region. + +Throughout the whole training process, we cannot adopt the same learning rate to update the weights, otherwise, the optimal point cannot be reached, So we need to adjust the learning rate during the training. In the initial stage of training, since the weights are in a random initialization state and the loss function decreases fast, a larger learning rate can be set. And in the later stage of training, since the weights are close to the optimal value, a larger learning rate cannot further find the optimal value, so a smaller learning rate needs is a better choice. As for the learning rate decay strategy, many researchers or practitioners use piecewise_decay (step_decay), which is a stepwise decay learning rate. In addition, there are also other methods proposed by researchers, such as polynomial_decay, exponential_ decay, cosine_decay, etc. Among them, cosine_decay requires no adjustment of hyperparameters and has higher robustness, thus emerging as the preferred learning rate decay method to improve model accuracy. + +The learning rates of cosine_decay and piecewise_decay are shown in the following figure. It is easy to observe that cosine_decay keeps a large learning rate throughout the training, so it is slow in convergence, but its final effect is better than peicewise_decay. + +![](../../images/models/lr_decay.jpeg) + +In addition, it is also observed that only a few rounds in cosine_decay use a small learning rate, which affects the final accuracy. So it is recommended to iterate more rounds for better results. + +Finally, when training a neural network with a large batch_size, it is recommended to use the warmup strategy, which, as the name implies, is a warm-up for the learning rate with no direct adoption of maximum learning rate at the beginning of training, but to train the network with a gradually increasing rate, and then decay the learning rate when it peaks. Experiments show that warmup can steadily improve the accuracy of the model when the batch_size is large. Specifically for the dataset. The specific information of the dataset is as follows: + +- ImageNet-1k: The recommended batch-size is 256, the initial learning rate is 0.1, and the cosine-decay to decrease the learning rate. +- Other datasets (ImageNet-1k pre-training by default): the larger the dataset, the larger the initial learning rate, not exceeding 0.1 for the best (when the batch-size is 256); the smaller the dataset, the smaller the initial learning rate, when the dataset is small, the use of warmup will also bring some accuracy improvement, and cosine-decay is still recommended as the learning rate decay strategy. + +### Q5.3 How to set the batch-size? + +**A**: + +Batch_size is an important hyperparameter in neural networks training, whose value determines how much data is fed into the neural network for training at a time. Previous researchers have experimentally found that when the value of batch_size is linearly related to the value of learning rate, the convergence accuracy is almost unaffected. When training ImageNet-1k data, most of the neural networks choose an initial learning rate of 0.1 and a batch_size of 256. The specific information of the dataset is as follows: + +- ImageNet-1k: learning rate is set to 0.1*k, batch_size is set to 256*k. +- Other datasets (ImageNet-1k pre-training by default): can be set according to the actual situation (e.g. smaller learning rate), but when adjusting the learning rate or batch size, another value should be adjusted at the same time. + +### Q5.4 What is weight_decay? How to set it? + +**A**: + +Overfitting is a common term in machine learning, which is simply understood as a model that performs well on training data but less satisfactory on test data. In image classification, there is also the problem of overfitting, and many regularization methods are proposed to avoid it, among which weight_decay is one of the widely used ways. When using SGD optimizer, weight_decay is equivalent to adding L2 regularization after the final loss function, which makes the weights of the network tend to choose smaller values, so eventually, the parameter values in the whole network tend to be more towards 0, and the generalization performance of the model is improved accordingly. In the implementation of major deep learning frameworks, this value means the coefficient before the L2 regularization, which is called L2Decay in the PaddlePaddle framework. The larger the coefficient is, the stronger the added regularization is, and the more the model tends to be underfitted. The specific information of the dataset is as follows: + +- ImageNet-1k: Most networks set the value of this parameter to 1e-4, and in some smaller networks such as the MobileNet series network, the value is set between 1e-5 and 4e-5 to avoid the underfitting. The following table shows the accuracy of MobileNetV1_x0_25 on ImageNet-1k using different L2Decay. Since MobileNetV1_x0_25 is a relatively small network, an overly large L2Decay will tend to underfit the network, so 3e-5 is a better choice in this network compared to 1e-4. + +| Model | L2Decay | Train acc1/acc5 | Test acc1/acc5 | +| ----------------- | ------- | --------------- | -------------- | +| MobileNetV1_x0_25 | 1e-4 | 43.79%/67.61% | 50.41%/74.70% | +| MobileNetV1_x0_25 | 3e-5 | 47.38%/70.83% | 51.45%/75.45% | + +In addition, the setting of this value is also related to whether other regularizations are used during training. If the data preprocessing is complicated, which equates with a harder training task, the value can be reduced appropriately. The following table shows the accuracy of ResNet50 on ImageNet-1k using different L2Decay after the RandAugment preprocessing. It is easy to observe that a smaller l2_decay helps to improve the model accuracy for a harder task. + +| Model | L2Decay | Train acc1/acc5 | Test acc1/acc5 | +| -------- | ------- | --------------- | -------------- | +| ResNet50 | 1e-4 | 75.13%/90.42% | 77.65%/93.79% | +| ResNet50 | 7e-5 | 75.56%/90.55% | 78.04%/93.74% | + +- Other datasets (ImageNet-1k pre-training loaded by default): When transferring, it is better not to change the value of L2Decay when training ImageNet-1k (i.e. training to get the L2Decay value of pre-training, the L2Decay value of each backbone is in the corresponding yaml file), and the change of learning rate is enough for general datasets. + +### Q5.5 Should I use label_smoothing and how to set the value of the parameter? + +**A**: + +Label_smoothing is a regularization method in deep learning, whose full name is Label Smoothing Regularization (LSR). In the traditional classification task, the loss function is calculated by the cross-entropy of the real one hot label and the output of the neural network, while label_smoothing is a label smoothing of the real one hot label, so that the label learned by the network is no longer a hard label, but a soft label with a probability value, where the probability at the position corresponding to the category is the largest and others small. In label_smoothing, the epsilon parameter describes the degree of label softening, the larger the value, the smaller the label probability value of the label vector after label smoothing, the smoother the label, and vice versa. The specific information of the dataset is as follows: + +- ImageNet-1k: This value is usually set to 0.1 in experiments training ImageNet-1k, and there is a steady increase in accuracy for models of the ResNet50 size and above after using label_smooting. The following table shows the accuracy metrics of ResNet50_vd before and after using label_smoothing. + +| Model | Use_label_smoothing(0.1) | Test acc1 | +| ----------- | ------------------------ | --------- | +| ResNet50_vd | 0 | 77.9% | +| ResNet50_vd | 1 | 78.4% | + +At the same time, since label_smoohing can be regarded as a regularization method, the accuracy improvement is not obvious or even decreases on a relatively small model. The following table shows the accuracy metrics of ResNet18 before and after using label_smoothing on ImageNet-1k. It is clear that the accuracy drops after using label_smoothing. + +| Model | Use_label_smoohing(0.1) | Train acc1/acc5 | Test acc1/acc5 | +| -------- | ----------------------- | --------------- | -------------- | +| ResNet18 | 0 | 69.81%/87.70% | 70.98%/89.92% | +| ResNet18 | 1 | 68.00%/86.56% | 70.81%/89.89% | + +Here is a trick to make label-smoothing effective even in smaller models, i.e., a fully connected layer of size 1000-2000 after the Global-Average-Pool, which works better with label-smoothing. + +- Other datasets (loaded with ImageNet-1k pre-training by default): the use of label-smooth tends to improve the accuracy, the smaller the dataset the larger the epsilon value can be. And in some smaller fine-grained images, the best model is usually obtained with the value set to 0.4-0.5. + +### Q5.6 Is random-crop still adjustable in the default image preprocessing? How ? + +**A**: + +In the standard preprocessing of ImageNet-1k data, the random_crop function defines two values, scale and ratio, which respectively determine the size of the image crop and the degree of image stretching, where the default value of the former is 0.08-1 (lower_scale-upper_scale), and the latter is 3/4-4/3 (lower_ratio-upper_ratio). In very small networks, this kind of data augmentation can lead to network underfitting and decreased accuracy. To the end, the data augmentation can be made weaker by increasing the crop area of the image or decreasing the stretching of the image. Weaker image transformation can be achieved by increasing the value of lower_scale or reducing the difference between lower_ratio and upper_scale, respectively. The specific information of the dataset is as follows: + +- ImageNet-1k: It is recommended to use only the default value for networks that are not particularly small, and to increase the value of lower_scale (to increase the crop area) or decrease the range of ratio values (to weaken the image stretching) for networks that are particularly small, and conduct the opposite for networks that are particularly large. The following table shows the accuracy of training MobileNetV2_x0_25 with different lower_scale, and we can see that the training accuracy and verification accuracy are improved by increasing the crop area of the images. + +| Model | Range of Scale | Train_acc1/acc5 | Test_acc1/acc5 | +| ----------------- | -------------- | --------------- | -------------- | +| MobileNetV2_x0_25 | [0.08,1] | 50.36%/72.98% | 52.35%/75.65% | +| MobileNetV2_x0_25 | [0.2,1] | 54.39%/77.08% | 53.18%/76.14% | + +- Other datasets (ImageNet-1k pre-training is loaded by default): it is recommended to use the default value, if the overfitting is too over, please consider adjusting the value of lower_scale (to reduce the crop area) or increasing the range of ratio values (to enhance the image stretching). + +### Q5.7 What are the common data augmentation? How to choose? + +**A**: + +In general, the size of the dataset is crucial to the performance, but the annotation of images is often expensive, hence there are rare annotated images, which highlight the importance of data augmentation. In the standard data augmentation for training ImageNet-1k, two methods, Random_Crop and Random_Flip, are mainly adopted. However, in recent years, an increasing number of data augmentation methods have been proposed, such as cutout, mixup, cutmix, AutoAugment, etc. Experiments show that these methods can effectively improve the accuracy of the model. The specific information of the dataset is as follows: + +- ImageNet-1k: The following table lists the performance of ResNet50 adopting 8 different data augmentation methods. It can be seen that all of them are beneficial compared to baseline, with cutmix being the most effective data augmentation so far. For more information about data augmentation, please refer to the chapter of [**Data Augmentation**](../advanced_tutorials/DataAugmentation_en.md). + +| Model | Date Augmentation Method | Test top-1 | +| -------- | ------------------------ | ---------- | +| ResNet50 | Standard Transformation | 77.31% | +| ResNet50 | Auto-Augment | 77.95% | +| ResNet50 | Mixup | 78.28% | +| ResNet50 | Cutmix | 78.39% | +| ResNet50 | Cutout | 78.01% | +| ResNet50 | Gridmask | 77.85% | +| ResNet50 | Random-Augment | 77.70% | +| ResNet50 | Random-Erasing | 77.91% | +| ResNet50 | Hide-and-Seek | 77.43% | + +- Other datasets (ImageNet-1k pre-training is loaded by default): In other datasets except those using Auto-Augment, there is generally an accuracy gain. Auto-Augment will search for each dataset with an independent hyper-parameter which determines how the data is processed, so the default ImageNet-1k hyper-parameter is not suitable for all datasets, but you can use Random-Augmentation instead of Auto-Augmentation. Other strategies can be used normally, but for harder tasks or smaller networks, it is not recommended to adopt strong data augmentation. + +In addition, multiple data augmentations can be overlaid to further improve accuracy when the data set is simple or the data size is small. + +### Q5.8 How to determine the tuning strategy by train_acc and test_acc? + +**A**: + +In the process of training a network, the accuracy of the training set and validation set are usually printed for each epoch, which portrays the performance of the model on both datasets. Generally speaking, the training set accuracy reflects the data accuracy after Random-Crop, and since the data is often more complex after Random-Crop, the training set accuracy and the validation set accuracy are often not the same concepts. + +- ImageNet-1k: Generally speaking, it is good to have a comparable accuracy or a slightly higher accuracy in the training set than in the validation set. If we find that the accuracy of the training set is much higher than the validation set, it means that the training set is overfitted and we need to add more regularity, such as increasing the value of L2Decay, adding more data augmentation strategies, introducing label_smoothing strategies, etc. If we find that the accuracy of the training set is lower than the validation set, it means that the training set is probably underfitted, and the regularization effect should be weakened during the training, such as reducing the value of L2Decay, decreasing the data augmentation methods, increasing the area of the crop area, weakening the image stretching, removing label_smoothing, etc. +- Other datasets (ImageNet-1k pretraining loaded by default): basically the same as ImageNet-1k training, in addition, if the model on other datasets tends to overfit (train acc much larger than test acc), you can also use better pretraining weights. PaddleClas provides distillation pretraining weights of SSLD for common networks, which are better than those of ImageNet-1k, expecting your preference. +- **[Note]** It is not recommended to readjust the training strategy according to the loss. After using different data augmentation, the size of the train loss varies greatly. For example, after using Cutmix or RandAugmentation, the train loss will exceed the test loss, and when the data augmentation strategy is weakened, the train loss will be smaller than the test loss, making it more difficult to adjust. + +### Q5.9 How to improve the accuracy of your own dataset by pre-training the model? + +**A**: + +At this stage, it has become a common practice in the image recognition field to load pre-trained models to train their own tasks, which can often improve the accuracy of a particular task compared to training from random initialization. In general, the pre-training model widely used in the industry is obtained by training the ImageNet-1k dataset of 1.28 million images of 1000 classes. The fc layer weights of this pre-training model are a matrix of k*1000, where k is the number of neurons before the fc layer, and it is not necessary to load the fc layer weights when loading the pre-training weights. In terms of the learning rate, if your dataset is particularly small (e.g., less than 1,000), we recommend you to adopt a small initial learning rate, e.g., 0.001 (batch_size:256, the same below), so as not to corrupt the pre-training weights with a larger learning rate. If your training dataset is relatively large (>100,000), we suggest you try a larger initial learning rate, such as 0.01 or above. If the target dataset is small, you can also freeze some shallow weights. Also, if you want to train a small dataset for a specific vertical class, you can train a pre-training weight on a related large dataset first, and then fine-tune the model with a smaller learning rate on that weight. + +### Q5.10 Existing strategies have saturated the accuracy of the model, how can the accuracy of a particular model be further improved? + +**A**: If the existing strategy cannot further improve the accuracy of the model, it means that the model has almost reached saturation with the existing dataset and strategy, and two methods are provided here. + +- Mining relevant data: Use the model trained on the existing dataset to make predictions on the relevant data, label the data with higher confidence and add it to the training set for further training. Repeat the steps above to further improve the accuracy of the model. +- Knowledge distillation: You can use a larger model to train a teacher model with higher accuracy on the dataset, and then adopt the teacher model to teach a Student model, where the Student model is the target model. PaddleClas provides Baidu's own SSLD knowledge distillation scheme, which can steadily improve by more than 3% even on such a challenging classification task as ImageNet-1k. For the chapter on SSLD knowledge distillation, please refer to [**SSLD Knowledge Distillation**](../advanced_tutorials/knowledge_distillation_en.md). + + + +## Issue 6 + +### Q6.1: What are the differences between the several branches of PaddleClas? How should I choose? + +**A**: PaddleClas currently has 3 branches: + +- Develop: develop branch is the development branch of PaddleClas as well as the most updated branch. All new features and changes will proceed on this branch first. If you want to keep track of the latest progress of PaddleClas, you can follow this branch. This branch mainly supports dynamic graphs and will be updated along with the version of paddlepaddle. +- Stable release (e.g. release/2.1.3): Fast updates keep followers informed of the latest progress, but they can also bring instability. Therefore, at critical points, we pull branches from the develop branch to provide stable releases, and the latest stable branch is also the default branch. Note that we only maintain the latest stable release, and generally fix bugs only without updating new features and models unless there are special circumstances. +- Static branch is a branch mainly for old users that adopts the static version of the graph, receiving simple maintenance only with no new features and models updated. It is not recommended for new users, and those who still employ it is recommended to turn to the dynamic graph branch or the stable release branch if the condition permits. + +In general, it is recommended to choose the develop branch if you want to keep up with the latest developments of PaddleClas, and the latest stable release branch if you need a stable version. + +### Q6.2: What is the static graph mode? + +**A**: + +The static graph mode is declarative programming, which is initially adopted by many deep learning frameworks such as tensorflow, mxnet, etc. In the static graph mode, you need to define the model structure first, and then the framework will compile and optimize the model structure to build the "computational graph". It can be simply understood that the mode is a static and unchanging pattern of the "computational graph". The advantage of it is that the compiler generally builds the graph once, which is relatively efficient, but the disadvantage lies in that it is not flexible enough and troublesome to debug. For example, if you run a static graph model in paddle, you need to complete all the operations, then extract the output according to a specific key, which means you cannot get the result in real-time. + +### Q6.3: What is the dynamic graph mode? + +**A**: + +Dynamic diagram mode is imperative programming, where users do not need to pre-define the network structure, and each line of code can be run directly to get the result. Compared with static graph mode, this one is more user-friendly and easier to debug. In addition, the structure design of dynamic graph mode is more flexible, and the structure can be adjusted at any time during the operation. + +PaddleClas currently uses a dynamic graph model for its continuously updated develop branch and the stable release branch. If you are new, it is recommended to use dynamic graph mode for development and training. If there is a performance requirement for inference and prediction, you can convert the dynamic graph model to a static one to improve efficiency after the training. + +### Q6.5: When building a classification dataset, how to build the data of the "background" category? + +**A**: + +In practice, it is often necessary to construct your own classification dataset for training purposes. In addition to the required category data, an additional category is needed, i.e., a "background" category. For example, if we create a cat and dog classification, with cats as one category and dogs as another, then an input image of a rabbit will be forced into one of these two categories. Therefore, during training, we should add some data from non-target categories as the "background" category data. + +When constructing the data for the "background" category, the first step is to consider the actual requirements. For example, if the actual test data are all animals, the "background" category data should include some animals other than dogs and cats. If the test data contains more categories, such as a tree, then the "background" category should be enriched. To put it simply, the data in the "background" category should be collected according to the situations that may occur in the actual scenario. The more situations included, the more types of data need to be covered, and the more difficult the task will be. Therefore, in practice, it is better to limit the problem to avoid the waste of resources and computing power. diff --git a/src/PaddleClas/docs/en/faq_series/faq_2021_s1_en.md b/src/PaddleClas/docs/en/faq_series/faq_2021_s1_en.md new file mode 100644 index 0000000..c730f25 --- /dev/null +++ b/src/PaddleClas/docs/en/faq_series/faq_2021_s1_en.md @@ -0,0 +1,267 @@ +# Image Classification FAQ Summary - 2021 Season 1 + +## Catalogue + +- [1. Issue 1](#1)(2021.01.05) +- [2. Issue 2](#2)(2021.01.14) +- [3. Issue 3](#3)(2020.01.21) +- [4. Issue 4](#4)(2021.01.28) +- [5. Issue 5](#5)(2021.02.03) + + + +## Issue 1 + +### Q1.1: Why is the prediction accuracy of the exported inference model very low ? + +**A**：You can check the following aspects: + +- Check whether the path of the pre-training model is correct or not. +- The default class number is 1000 when exporting the model. If the pre-training model has a custom class number, you need to specify the parameter `--class_num=k` when exporting, k is the custom class number. +- Compare the output class id and score of `tools/infer/infer.py` and `tools/infer/predict.py` for the same input. If they are exactly the same, the pre-trained model may have poor accuracy itself. + +### Q1.2: How to deal with the unbalanced categories of training samples? + +**A**：There are several commonly used methods. + +- From the perspective of sampling + - The samples can be sampled dynamically according to the categories, with different sampling probabilities for each category and ensure that the number of training samples in different categories is basically the same or in the desired proportion in the same minibatch or epoch. + - You can use the oversampling method to oversample the categories with a small number of images. +- From the perspective of loss function + - The OHEM (online hard example miniing) method can be used to filter the hard example based on the loss of the samples for gradient backpropagation and parameter update of the model. + - The Focal loss method can be used to assign a smaller weight to the loss of some easy samples and a larger weight to the loss of hard samples, so that the loss of easy samples contributes to the overall loss of the network without dominating the loss. + +### Q1.3 When training in docker, the data path and configuration are fine, but it keeps reporting `SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception`, why is this? + +**A**： + +This may be caused by the small shared memory in docker. When creating docker, the default size of `/dev/shm` is 64M, if you use multiple processes to read data, the shared memory may fall short, so you need to allocate more space to `/dev/shm`. When creating a docker, input `--shm-size=8g` to allocate 8G to `/dev/shm`, which is usually enough. + +### Q1.4 Where can I download the 10W class image classification pre-training model provided by PaddleClas and how to use it? + +**A**： + +Based on ResNet50_vd, Baidu open-sourced its own large-scale classification pre-training model with 100,000 categories and 43 million images. The former is available for download at [download address](https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_10w_pretrained.tar), where it should be noted that the pre-training model does not provide the final FC layer parameters and thus cannot be used directly for inference; however, it can be used as a pre-training model to fine-tune it on your own dataset. It is verified that this pre-training model has a more significant accuracy gain of up to 30% on different datasets than the ResNet50_vd pre-training model based on the ImageNet1k dataset. + +### Q1.5 How to accelerate when using C++ for inference deployment? + +**A**：You can speed up the inference in the following ways. + +1. For CPU inference, you can turn on mkldnn and increase the number of threads (cpu_math_library_num_threads, in `tools/config.txt`), which is usually 6~10. +2. For GPU inference, you can enable TensorRT inference and FP16 inference if the hardware condition allows, which can further speed up the speed. +3. If the memory or video memory is sufficient, the batch size of inference can be increased. +4. The image preprocessing logic (mainly designed for resize, crop, normalize, etc.) can be run on the GPU, which can further speed up the process. + +You are welcomed to add more tips on inference deployment acceleration. + + + +## Issue 2 + +### Q2.1: Does PaddleClas have to start from 0 when setting labels, and does class_num have to equal the number of classes in the dataset? + +**A**： + +In PaddleClas, the label start from 0 by default, so try to set the label from 0. Of course, it is possible to set them from other values, which will result in a larger class_num and will in turn lead to a larger number of FC layer parameters for the classification, so the weight file will take up more storage space. In the case of a continuous set of classes, set class_num equal to the number of classes in the dataset (of course, it is acceptable to set it greater than the number of classes in the dataset, and even higher accuracy can be obtained in many datasets, but it will also result in a larger number of FC layers), and in the case of a discontinuous set of classes, the class_num should be equal to the largest class_id+1 in the dataset. + +### Q2.2: How to address the the problem of large space occupation of weight file resulted from great FC due to numerous class number? + +**A**： + +The final FC weight is a large matrix of size C*class_num, where C is the number of neural units in the previous layer of FC, e.g., C is 2048 in ResNet50, and the size of FC weight can be further reduced by decreasing the value of C. For example, a layer of FC with smaller dimension can be added after GAP, which can greatly reduce the weight size of the final classification layer. + +### Q2.3: Why did the training of ssld distillation on a custom dataset using PaddleClas fail to meet the expectation? + +First, it is necessary to ensure that the accuracy of the Teacher model. Second, it is necessary to ensure that the Student model is successfully loaded with the pre-training weights of ImageNet-1k and the Teacher model is successfully loaded with the weights of the training custom dataset. Finally, it is necessary to ensure that the initial learning rate is not too large, or at least smaller than the value of the ImageNet-1k training. + +### Q2.4: Which networks have advantages on mobile or embedded side? + +It is recommended to use the Mobile Series network, and the details can be found in [Introduction to Mobile Series Network Structure](../models/Mobile_en.md). If the speed of the task is the priority, MobileNetV3 series can be considered, and if the model size is more important, the specific structure can be determined based on the StorageSize - Accuracy in the Introduction to the Mobile Series Network Structure. + +### Q2.5: Why use a network with large number of parameters and computation such as ResNet when the mobile network is so fast? + +Different network structures have various speed advantages running on disparate devices. On the mobile side, mobile series networks run faster than server-side networks, but on the server side, networks with specific optimizations such as ResNet have greater advantages for the same accuracy. So the specific network structure needs to be decided on a case-by-case basis. + + + +## Issue 3 + +### Q3.1: What are the characteristics of a double (multi)-branch structure and a Plain structure, respectively? + +**A**： + +Plain networks, represented by VGG, have evolved into multi-branch network structures, represented by the ResNet series (with residual modules) and the Inception series (with multiple convolutional kernels in parallel). It is found that the multi-branch structure is more friendly in the model training, and the larger network width can bring stronger feature fitting ability, while the residual structure can avoid the problem of disappearing gradient of the deep network. However, in the inference phase, the model with a multi-branch structure has no speed advantage. Even though the FLOPs of the model are lower, the computational density of the multi-branch structure model is also dissatisfactory. For example, the FLOPs of VGG16 model are much larger than those of EfficientNetB3, but the inference speed of the latter is significantly faster than the former. Therefore, the multi-branch structure is more friendly in the model training, while the Plain structure model is more suitable for the inference. Starting from this, we can use a multi-branch network structure in the training phase to exchange a larger training time cost for a model with better feature fitting ability, and convert the multi-branch structure to Plain structure in the inference phase to exchange a shorter inference time. The conversion from multi-branch to a Plain structure can be achieved by the structural re-parameterization technique. + +In addition, the Plain structure is more friendly for pruning operations. + +Note: The term "Plain structure" and "structural re-parameterization" are from the paper "RepVGG: Making VGG-style ConvNets Great Again". Plain structure network model means that there is no branching structure in the whole network, i.e., the input of layer `i` is the output of layer `i-1` and the output of layer `i` is the input of layer `i+1`. + +### Q3.2: What are the main innovations of ACNet? + +**A**： ACNet means "Asymmetric Convolution Block", and the idea is from the paper "ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks", which proposes a set of three CNN convolutional kernels with "ACB" structure to replace the traditional square convolutional kernels in existing convolutional neural networks in the training phase. + +The size of the square convolution kernel is assumed to be `d*d`, i.e., the width and height are all `d`. The ACB structure used to replace this convolution kernel is three convolution kernels of size `d*d`, `1*d`, and `d*1`, and then the outputs of the three convolution kernels are added directly to obtain the same size as the original square convolution kernel. After the training, the ACB structure is replaced by the original square convolution kernel, and the parameters of which are directly added to the parameters of the three convolution kernels of the ACB structure (see `Q3.4`), so the same model structure as before is used for inference, and the ACB structure is only used in the training phase. + +During the training, the network width of the model is improved by the ACB structure, and more features are extracted using the two asymmetric convolution kernels of `1*d` and `d*1` to enrich the information of the feature maps extracted by the `d*d` convolution kernels. In the inference stage, this design idea does not bring additional parameters and computational overhead. The following figure shows the form of convolutional kernels for the training phase and the inference deployment phase, respectively. + +![](../../images/faq/TrainingtimeACNet.png) + +![](../../images/faq/DeployedACNet.png) + +Experiments by the authors of the article show that the model capability can be significantly improved by using ACNet structures in the training of the original network model, as explained by the original authors as follows. + +1. Experiments show that for a `d*d` convolution kernel, the parameters of the skeleton position (e.g., the `skeleton` position of the convolution kernel in the above figure) have a greater impact on the model accuracy than the parameters of the corner position (e.g., the `corners` position of the convolution kernel in the above figure) of the eliminated convolution kernel, so the parameters of the skeleton position of the convolution kernel are essential. And the two asymmetric convolution kernels in the ACB structure enhance the weight of the square convolution kernel skeleton position parameter, making it more significant. About whether this summation will weaken the role of the skeleton position parameter due to the offsetting effect of positive and negative numbers, the authors found through experiments that the training of the network always goes in the direction of increasing the role of the skeleton position parameter, and there is no weakening due to the offsetting effect. +2. The asymmetric convolution kernel is more robust for flipped images, as shown in the following figure, the horizontal asymmetric convolution kernel is more robust for flipped images up and down. The feature maps extracted by the asymmetric convolution kernel are the same for semantically the same position in the image before and after the flip, which is better than the square convolution kernel. + +![](../../images/faq/HorizontalKernel.png) + +### Q3.3: What are the main innovations of RepVGG? + +**A**： + +Through Q3.1 and Q3.2, it may occur to us to decouple the training phase and inference phase by ACNet, and use multi-branch structure in the training phase and Plain structure in inference phase, which is the innovation of RepVGG. The following figure shows the comparison of the network structures of ResNet and RepVGG in the training and inference phases. + +![](../../images/faq/RepVGG.png) + +First, the RepVGG in the training phase adopts a multi-branch structure, which can be regarded as a residual structure with `1*1` convolution and constant mapping on top of the traditional VGG network, while the RepVGG in the inference phase degenerates to a VGG structure. The transformation of the network structure from RepVGG in the training phase to RepVGG in the inference phase is implemented using the "structural reparameterization" technique. + +The constant mapping can be regarded as the output of the `1*1` convolution kernel with `1` parameters acting on the input feature map, so the convolution module of RepVGG in the training phase can be regarded as two `1*1` convolutions and one `3*3` convolution, and the parameters of the `1*1` convolution can be directly added to the parameters at the center of the `3*3` convolution kernel (this operation is similar to the operation of adding the parameters of the asymmetric convolution kernel to the parameters of the skeleton position of the square convolution kernel in ACNet). By doing this, the constant mapping, `1*1` convolution, and `3*3` convolution branches of the network structure can be combined into one `3*3` convolution in the inference stage, as described in `Q3.4`. + +### Q3.4: What are the similarities and differences between the struct re-parameters in ACNet and RepVGG? + +**A**： + +From the above, it can be simply understood that RepVGG is the extreme version of ACNet. Re-parameters operation in ACNet is shown in the following figure. + +![](../../images/faq/ACNetReParams.png) + +Take `conv2` as an example, the asymmetric convolution can be regarded as a square convolution kernel of `3*3`, except that the upper and lower six parameters of the square convolution kernel are `0`, and it is the same for `conv3`. On top of that, the sum of the results of `conv1`, `conv2`, and `conv3` is equivalent to the sum of three convolution kernels followed by convolution. With `Conv` denoting the convolution operation and `+` denoting the addition operation of the matrix, then: `Conv1(A)+Conv2(A)+Conv3(A) == Convk(A)`, where `Conv1`, ` Conv2`, `Conv3` have convolution kernels `Kernel1`, `kernel2`, `kernel3`, and `Convk` has convolution kernels `Kernel1 + kernel2 + kernel3`, respectively. + +The RepVGG network is the same as ACNet, except that the `1*d` asymmetric convolution of ACNet becomes the `1*1` convolution, and the position where the `1*1` convolutions are summed becomes the center of the `3*3` convolution. + +### Q3.5: What are the factors that affect the computation speed of a model? Does a model with a larger number of parameters necessarily have a slower computation speed? + +**A**： + +There are many factors that affect the computation speed of the model, and the number of parameters is only one of them. Specifically, without considering the hardware differences, the computation speed of the model can be referred to the following aspects. + +1. Number of parameters: the number of parameters used to measure the model, the larger the number of parameters of the model, the higher the memory (video memory) requirements of the model during computation. However, the size of the memory (video memory) footprint does not depend entirely on the number of parameters. In the figure below, assuming that the input feature map memory footprint size is `1` unit, for the residual structure on the left, the peak memory footprint during computation is twice as large as that of the Plain structure on the right, because the results of the two branches need to be recorded and then added together. + +![](../../images/faq/MemoryOccupation.png) + +2. FLOPs: Note that FLOPs are distinguished from floating point operations per second (FLOPS), which can be simply understood as the amount of computation and is usually adopted to measure the computational complexity of a model. Taking the common convolution operation as an example, considering no batch size, activation function, stride operation, and bias, assuming that the input future map size is `Min*Min` and the number of channels is `Cin`, the output future map size is `Mout*Mout` and the number of channels is `Cout`, and the conv kernel size is `K*K`, the FLOPs for a single convolution can be calculated as follows. + 1. The number of feature points contained in the output feature map is: `Cout * Mout * Mout`. + 1. For the convolution operation for each feature point in the output feature map: the number of multiplication calculations is: `Cin * K * K`; the number of addition calculations is: `Cin * K * K - 1`. + 1. So the total number of computations is: `Cout * Mout * Mout * (Cin * K * K + Cin * K * K - 1)`, i.e. `Cout * Mout * Mout * (2Cin * K * K - 1)`. + +3. Memory Access Cost (MAC): The computer needs to read the data from memory (general memory, including video memory) to the operator's Cache before performing operations on data (such as multiplication and addition), and the memory access is very time-consuming. Take grouped convolution as an example, suppose it is divided into `g` groups, although the number of parameters and FLOPs of the model remain unchanged after grouping, the number of memory accesses for grouped convolution becomes `g` times of the previous one (this is a simple calculation without considering multi-level Cache), so the MAC increases significantly and the computation speed of the model slows down accordingly. + +4. Parallelism: The term parallelism often includes data parallelism and model parallelism, in this case, model parallelism. Take convolutional operation as an example, the number of parameters in a convolutional layer is usually very large, so if the matrix in the convolutional layer is chunked and then handed over to multiple GPUs separately, the purpose of acceleration can be achieved. Even some network layers with too many parameters for a single GPU memory may be divided into multiple GPUs, but whether they can be divided into multiple GPUs in parallel depends not only on hardware conditions, but also on the specific form of operation. Of course, the higher the degree of parallelism, the faster the model can run. + + + + + +## Issue 4 + +### Q4.1: There is certain synthetic data in image classification task, is it necessary to use sample equalization? + +**A**: + +1. If the number of samples of different categories varies greatly, and the samples of one category are expanded to more than several times of other categories due to the synthetic data set, it is necessary to reduce the weights of that category appropriately. +2. If some categories are synthetic and some are semi-synthetic and semi-real, the equalization is not needed as long as the number is in an order of magnitude. Or you can try to train it, testing whether the synthetic category samples can be accurately identified. +3. If the performance of the category of different sources of data is degraded due to the increase of synthetic data, it is necessary to consider whether the synthetic dataset has noise or difficult samples. You can also properly increase the weight of the category to obtain better recognition performance of the category. + +### Q4.2: What new opportunities and challenges will be brought by the introduction of Vision Transformer (ViT) into the field of image classification by academia? What are the advantages over CNN? + +Paper address: [AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE](https://openreview.net/pdf?id=YicbFdNTTy) + +**A**: + +1. The dependence of images on CNNs is unnecessary, and the computational efficiency and scalability of the Transformer allow for training very large models without saturation as the model and dataset grow. Inspired by the Transformer for NLP, when being used in image classification tasks, images are divided into sequential patches that are fed into a linear unit embedding as input to the transformer. +2. In medium-sized datasets such as ImageNet1k, ImageNet21k, the visual Transformer model is several percentage points lower than ResNet of the same size. It is speculated that this is because the transformer lacks the Locality and Spatial Invariance that CNNs have, and it is difficult to outperform convolutional networks when the amount of data is not large enough. But for this problem, the data augmentation adopted by [DeiT](https://arxiv.org/abs/2012.12877) to some extent addresses the reliance of Vision Transformer on very large datasets for training. +3. This approach can go beyond local information and model more long-range dependencies when training on super large-scale datasets 14M-300M, while CNN can better focus on local information but is weak in capturing global information. +4. Transformer once reigned in the field of NLP, but was also questioned as not applicable to the CV field. The current several pieces of visual field articles also deliver competitive performance as the SOTA of CNN. We believe that a joint Vision-Language or multimodal model will be proposed that can solve both visual and linguistic problems. + +### Q4.3: For the Vision Transformer model, how is the image converted into sequence information for the Encoder? + +**A**: + +1. Using the Transformer model, mainly the attention approach. We want to conceive a scenario where semantic embedding information is applicable. But image classification is not very relevant to the semantic information of sequences, so Vision Transformer has its own unique design, and it is precisely the goal of ViT to use attention mechanisms instead of CNNs. +2. Consider the input form of the Encoder in Transformer, as shown below: + - (1) variable-length sequential input, because it is RNN structure with various amounts of words in one sentence. If it is an NLP scene, the change of word order affects little of the semantics, but the position of the image means a lot since great misunderstanding can be caused when different regions are connected in a different order. + - (2) Single patch position is transformed into a vector with fixed dimension. Encoder input is patch pixel information embedding, combined with some fixed position vector concate to synthesize a vector with fixed dimension and position information in it. + +![](../../images/faq/Transformer_input.png) + +3. Consider the following question: How to pass an image to an encoder? + +- As the following figure shows. Suppose the input image is [224,224,3], which is cut into many patches in order from left to right and top to bottom, and the patch size can be [p,p,3] (p can be 16, 32). Convert it into a feature vector using the Linear Projection of Flattened Patches module and concat a position vector into the Encoder. + +![](../../images/faq/ViT_structure.png) + +4. As shown above, given an image of `H×W×C` and a block size P, the image can be divided into `N` blocks of `P×P×C`, `N=H×W/(P×P)`. After getting the blocks, we have to use linear transformation to convert them into D-dimensional feature vectors, and then add the position encoding vectors. Similar to BERT, ViT also adds a classification flag bit before the sequence, denoted as `[CLS]`. The ViT input sequence `z` is shown in the following equation, where `x` represents an image block. + +![](../../images/faq/ViT.png) + +5. ViT model is basically the same as Transformer, where the input sequence is passed into ViT and then the final output features are classified using the `[CLS]` flags. viT consists mainly of MSA (multiheaded self-attentive) and MLP (two-layer fully connected network using GELU activation function), with LayerNorm and residual connections before MSA and MLP + +### Q4.4: How to understand Inductive Bias? + +**A**: + +1. In machine learning, some assumptions are made about the problem to be applied, and this assumption is called inductive preference. Certain a priori rules are inducted from the phenomena observed in real life, and then certain constraints are made on the model, thus playing the role of model selection. In CNN, it is assumed that the features have the characteristics of Locality and Spatial Invariance, that is, the adjacent features are connected but not those that are far away, and the adjacent features are fused together to produce "solutions" more easily; there is also the attention mechanism, which is the rule inducted from human intuition and life experience. +2. Vision Transformer utilizes an inductive bias that is linked to Sequentiality and Time Invariance, i.e., the time interval in sequence order, and therefore also yields better performance than CNN-like models on larger datasets. In the Conclusion of the article, "Unlike prior works using self-attention in computer vision, we do not introduce any image-specific inductive biases into the architecture" and "We find that large scale training trumps inductive bias" in the Introduction, we can conclude that intuitively the generation of inductive bias in the case of large amounts of data is performance-degrading and should be discarded whenever possible. + +### Q4.5: Why does ViT add a [CLS] flag bit? Why is the vector corresponding to the [CLS] used as the semantic representation of the whole sequence? + +**A**: + +1. Similar to BERT, ViT adds a `[CLS]` flag bit before the first patch, and the vector corresponding to the last end flag bit can be used as a semantic representation of the whole image, and thus for downstream classification tasks, etc. Therefore, the whole embedding group can characterize the features of the image at different locations. +2. The vector corresponding to the `[CLS]` flag bit is used as the semantic representation of the whole image because this symbol with no obvious semantic information will "fairly" integrate the semantic information of each patch in the image compared with other patches, and thus better represent the semantic of the whole image. + + + +## Issue 5 + +### Q5.1: What is included in the PaddleClas training profile? How to modify during the model training? + +**A**: PaddleClas configures 6 modules, namely: global configuration, network architecture, learning rate, optimizer, training and validation. + +The global configuration contains information about the configuration of the task, such as the number of categories, the amount of data in the training set, the number of epochs to train, the size of the network input, etc. If you want to train a custom task or use your own training set, please pay attention to this section. + +The configuration of the network structure defines the network to be used. In practice, the first step is to select the appropriate configuration file, so this part of the configuration is usually not modified. Modifications are only made when customizing the network structure, or when there are special requirements for the task. + +It is recommended to use the default configuration for the learning rate and optimizer. These parameters are the ones that are already tuned. Fine-tuning can also be done if the changes to the task are significant. + +The training and inference configurations include batch_size, dataset, data transforms (transforms), number of workers (num_workers) and other important configurations, which should be modified according to the actual environment. Note that the batch_size in paddleClas is a single card configuration, if it is a multi-card training, the total batch_size is a multiple of the one set in the configuration file. For example, if the configuration file has the batch_size of 64 and 4 cards trained, the total batch_size is 4*64=256. num_workers defines the number of processes on a single card, i.e., if num_workers is 8 with 4 cards for training, there are actually 32 workers. + +### Q5.2: How to quickly modify the configuration in the command line? + +**A**: + +During training, we often need to constantly fine-tune individual configurations without frequent modification of the configuration file. This can be adjusted using -o. The modification is done by first writing the name of the configuration to be changed by level, splitting the levels with dots, and then writing the value to be modified. For example, if we want to modify batch_size, we can add -o DataLoader.TRAIN.sampler.batch_size=512 after the training command. + +### Q5.3: How to choose the right model according to the accuracy curve of PaddleClas? + +**A**: + +PaddleClas provides benchmarks for several models and plots performance curves. There are mainly three kinds: accuracy-inference time curve, accuracy-parameter count curve and accuracy-FLOPS curve, with accuracy on the vertical axis and the other three on the horizontal axis. in general, different models perform consistently on the three plots. Models of the same series are represented on the plots using the same symbols and connected by curves. + +Taking the accuracy-inference time curve as an example, a higher point indicates higher accuracy, and a left one indicates faster speed. For example, the model in the upper-left region is a fast and accurate model, while the leftmost point close to the vertical axis is a lightweight model. When using this, you can choose the right model by considering both accuracy and time. As an example, if we want the most accurate model that runs under 10ms, first draw a vertical line 10ms out from the horizontal axis, and then find the highest point on the left side, which is the model that meets the requirements. + +In practice, the number of parameters and FLOPS of the model are constant, while the operation time varies under different hardware and software conditions. If you want to choose the model more accurately, then you can run the test in your own environment and get the corresponding performance graph. + +### Q5.4: If I want to add two classes in imagenet, can I fix the parameters of the existing fully connected layer and only train the new two? + +**A**: + +This idea works in theory, but I am afraid it will not work too well. If only the fully-connected layers are fixed and the parameters of the preceding convolutional layers are changed, there is no guarantee that these fully-connected layers will work the same as they did at the beginning. If the parameters of the entire network are kept constant and only the two new categories of fully connected layers are trained, it is also difficult to train the desired results. + +If you really need the original 1000 categories to be accurate, you can add the data of the new categories to the original training set, and then finetune with the pre-training model. if you only need a few of the 1000 categories, you can pick out this part of the data, and finetune it after mixing it with the new data. + +### Q5.5: When using classification models as pre-training models for other tasks, which layers should be selected as features? + +**A**: + +There are many strategies to use classification models as the backbone for other tasks, and a more basic approach is presented here. First, remove the final fully-connected layer, which mainly contains the classification information of the original task. If the task is relatively simple, just use the output of the previous layer as featuremap and add the structure corresponding to the task on top of it. If the task involves multiple scales, and the anchor of different scales needs to be selected, such as some detection models, then the output of the layer before each downsampling can be selected as the featuremap. diff --git a/src/PaddleClas/docs/en/faq_series/faq_2021_s2_en.md b/src/PaddleClas/docs/en/faq_series/faq_2021_s2_en.md new file mode 100644 index 0000000..c18da5b --- /dev/null +++ b/src/PaddleClas/docs/en/faq_series/faq_2021_s2_en.md @@ -0,0 +1,356 @@ +# PaddleClas FAQ Summary - 2021 Season 2 + +## Before You Read + +- We collect some frequently asked questions in issues and user groups since PaddleClas is open-sourced and provide brief answers, aiming to give some reference for the majority to save you from twists and turns. +- There are many talents in the field of image classification, recognition and retrieval with quickly updated models and papers, and the answers here mainly rely on our limited project practice, so it is not possible to cover all facets. We sincerely hope that the man of insight will help to supplement and correct the content, thanks a lot. + +## Catalogue + +- [1. Theory](#1) + - [1.1 Basic Knowledge of PaddleClas](#1.1) + - [1.2 Backbone Network and Pre-trained Model Library](#1.2) + - [1.3 Image Classification](#1.3) + - [1.4 General Detection](#1.4) + - [1.5 Image Recognition](#1.5) + - [1.6 Vector Search](#1.6) +- [2. Practice](#2) + - [2.1 Common Problems in Training and Evaluation](#2.1) + - [2.2 Image Classification](#2.2) + - [2.3 General Detection](#2.3) + - [2.4 Image Recognition](#2.4) + - [2.5 Vector Search](#2.5) + - [2.6 Model Inference Deployment](#2.6) + + + +## 1. Theory + + + +### 1.1 Basic Knowledge of PaddleClas + +#### Q1.1.1 Differences between PaddleClas and PaddleDetection + +**A**：PaddleClas is an image recognition repo that integrates mainbody detection, image classification, and image retrieval to solve most image recognition problems. It can be easily adopted by users to solve small sample and multi-category issues in the field. PaddleDetection provides the ability of target detection, keypoint detection, multi-target tracking, etc., which is convenient for users to locate the points and regions of interest in images, and is widely used in industrial quality inspection, remote sensing image detection, unmanned inspection and other projects. + +#### Q1.1.3: What does the parameter momentum mean in the Momentum optimizer? + +**A**: + +Momentum optimizer is based on SGD optimizer and introduces the concept of "momentum". In the SGD optimizer, the update of the parameter `w` at the time `t+1` can be expressed as + +``` +w_t+1 = w_t - lr * grad +``` + +`lr` is the learning rate and `grad` is the gradient of the parameter `w` at this point. With the introduction of momentum, the update of the parameter `w` can be expressed as + +``` +v_t+1 = m * v_t + lr * grad +w_t+1 = w_t - v_t+1 +``` + +Here `m` is the `momentum`, which is the weighted value of the cumulative momentum, generally taken as `0.9`. And when the value is less than `1`, the earlier the gradient is, the smaller the impact on the current. For example, when the momentum parameter `m` takes `0.9`, the weighted value of the gradient of `t-5` is `0.9 ^ 5 = 0.59049` at time `t`, while the value at time `t-2` is `0.9 ^ 2 = 0.81`. Therefore, it is intuitive that gradient information that is too "far away" is of little significance for the current reference, while "recent" historical gradient information matters more. + +[](../../images/faq/momentum.jpeg) + +By introducing the concept of momentum, the effect of historical updates is taken into account in parameter updates, thus speeding up the convergence and improving the loss (cost, loss) oscillation caused by the `SGD` optimizer. + +#### Q1.1.4: Does PaddleClas have an implementation of the paper `Fixing the train-test resolution discrepancy`? + +**A**: Currently, it is not implemented. If needed, you can try to modify the code yourself. In brief, the idea proposed in this paper is to fine-tune the final FC layer of the trained model using a larger resolution as input. Specifically, train the model network on a lower resolution dataset first, then set the parameter `stop_gradient=True ` for the weights of all layers of the network except the final FC layer, and at last fine-tune the network with a larger resolution input. + + + +### 1.2 Backbone Network and Pre-trained Model Library + + + +### 1.3 Image Classification + +#### Q1.3.1: Does PaddleClas provide data enhancement for adjusting image brightness, contrast, saturation, hue, etc.? + +**A**： + +PaddleClas provides a variety of data augmentation methods, which can be divided into 3 categories. + +1. Image transformation ： AutoAugment, RandAugment; +2. Image cropping： CutOut、RandErasing、HideAndSeek、GridMask； +3. Image aliasing ：Mixup, Cutmix. + +Among them, RandAngment provides a variety of random combinations of data augmentation methods, which can meet the needs of brightness, contrast, saturation, hue and other aspects. + + + +### 1.4 General Detection + +#### Q1.4.1 Does the mainbody detection only export one subject detection box at a time? + +**A**：The number of outputs for the main body detection is configurable through the configuration file. In the configuration file, Global.threshold controls the threshold value for detection, so boxes smaller than this threshold are discarded; and Global.max_det_results controls the maximum number of results returned. The two together determine the number of output detection boxes. + +#### Q1.4.2 How is the data selected for training the mainbody detection model? Will it harm the accuracy to switch to a smaller model? + +**A**： + +The training data is a randomly selected subset of publicly available datasets such as COCO, Object365, RPC, and LogoDet. We are currently introducing an ultra-lightweight mainbody detection model in version 2.3, which can be found in [Mainbody Detection](../../en/image_recognition_pipeline/mainbody_detection_en.md#2-model-selection). + +#### Q1.4.3: Is there any false detections in some scenarios with the current mainbody detection model? + +**A**：The current mainbody detection model is trained using publicly available datasets such as COCO, Object365, RPC, LogoDet, etc. If the data to be detected is similar to industrial quality inspection and other data with large differences from common categories, it is necessary to fine-tune the training based on the current detection model again. + + + +### 1.5 Image Recognition + +#### Q1.5.1 Is `triplet loss` needed for `circle loss` ? + +**A**： + +`circle loss` is a unified form of sample pair learning and classification learning, and `triplet loss` can be added if it is a classification learning. + +#### Q1.5.2 Which recognition model is better if not to recognize open source images in all four directions? + +**A**： + +The product recognition model is recommended. For one, the range of products is wider and the probability that the recognized image is a product is higher. For two, the training data of the product recognition model uses 50,000 categories of data, which has better generalization ability and more robust features. + +#### Q1.5.3 Why is 512-dimensional vector is finally adopted instead of 1024 or others? + +**A**： + +Vectors with small dimensions should be adopted. 128 or even smaller are practically used to speed up the computation. In general, a dimension of 512 is large enough to adequately represent the features. + + + +### 1.6 Vector Search + +#### Q1.6.1 Does the Möbius vector search algorithm currently used by PaddleClas support index.add() similar to the one used by faiss? Also, do I have to train every time I build a new graph? Is the train here to speed up the search or to build similar graphs? + +**A**：The faiss retrieval module is now supported in the release/2.3 branch and is no longer supported by Möbius, which provides a graph-based algorithm that is similar to the nearest neighbor search and currently supports two types of distance calculation: inner product and L2 distance. However, Möbius does not support the index.add function provided in faiss. So if you need to add content to the search library, you need to rebuild a new index from scratch. The search algorithm internally performs a train-like process each time the index is built, which is different from the train interface provided by faiss. Therefore, if you need the faiss module, you can use the release/2.3 branch, and if you need Möbius, you need to fall back to the release/2.2 branch for now. + +#### Q1.6.2: What exactly are the `Query` and `Gallery` configurations used for in the PaddleClas image recognition for Eval configuration file? + +**A**: + + Both `Query` and `Gallery` are data set configurations, where `Gallery` is used to configure the base library data and `Query` is used to configure the validation set. When performing Eval, the model is first used to forward compute feature vectors on the `Gallery` base library data, which are used to construct the base library, and then the model forward computes feature vectors on the data in the `Query` validation set, and then computes metrics such as recall rate in the base library. + + + +## 2. Practice + + + +### 2.1 Common Problems in Training and Evaluation + +#### Q2.1.1 Where is the `train_log` file in PaddleClas? + +**A**：`train.log` is stored in the path where the weights are stored. + +#### Q2.1.2 Why nan is the output of the model training? + +**A**： 1. Make sure the pre-trained model is loaded correctly, the easiest way is to add the parameter `-o Arch.pretrained=True`; 2. When fine-tuning the model, the learning rate should not be too large, e.g. 0.001. + +#### Q2.1.3 Is it possible to perform frame-by-frame prediction in a video? + +**A**：Yes, but currently PaddleClas does not support video input. You can try to modify the code of PaddleClas or store the video frame by frame before using PaddleClas. + +#### Q2.1.4: In data preprocessing, what setting can be adopted without cropping the input data? Or how to set the size of the crop? + +**A**: The data preprocessing operators supported by PaddleClas can be viewed in `ppcls/data/preprocess/__init__.py`, and all supported operators can be configured in the configuration file. The name of the operator needs to be the same as the operator class name, and the parameters need to be the same as the constructor parameters of the corresponding operator class. If you do not need to crop the image, you can remove `CropImage` and `RandCropImage` and replace them with `ResizeImage`, and you can set different resize methods with its parameters by using the `size` parameter to directly scale the image to a fixed size, and using the `resize_short` parameter to maintain the image aspect ratio for scaling. To set the crop size, use the `size` parameter of the `CropImage` operator, or the `size` parameter of the `RandCropImage` operator. + +#### Q2.1.5: Why do I get a usage error after PaddlePaddle installation and cannot import any modules under paddle (import paddle.xxx)? + +**A**: + +You can first test if Paddle is installed correctly by using the following code. + +``` +import paddle +paddle.utils.install_check.run_check(） +``` + +When installed correctly, the following prompts will be displayed. + +``` +PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now. +``` + +Otherwise, the relevant question will prompt out. Also, after installing both the CPU and the GPU version of Paddle, you will need to uninstall both versions and reinstall the required version due to conflicts between the two versions. + +#### Q2.1.6: How to save the optimal model during training? + +**A**: + +PaddleClas saves/updates the following three types of models during training. + +1. the latest model (`latest.pdopt`, `latest.pdparams`, `latest.pdstates`), which can be used to resume training when it is unexpectedly interrupted. +2. the best model (`best_model.pdopt`, `best_model.pdparams`, `best_model.pdstates`). +3. breakpoints at the end of an epoch during training (`epoch_xxx.pdopt`, `epoch_xxx.pdparams`, `epoch_xxx.pdstates`). The `Global.save_interval` field in the training profile indicates the save interval for this model. If you make it larger than the total number of epochs, intermediate breakpoint models will no longer be saved. + +#### Q2.1.7: How to address the `ERROR: Unexpected segmentation fault encountered in DataLoader workers.` during training? + +**A**：Try setting the field `num_workers` in the training profile to `0`; try making the field `batch_size` in the training profile smaller; ensure that the dataset format and the dataset path in the profile are correct. + +#### Q2.1.8: How to use `Mixup` and `Cutmix` during training? + +**A**： + +- For `Mixup`, please refer to [Mixup](../../../ppcls/configs/ImageNet/DataAugment/ResNet50_ Mixup.yaml#L63-L65); and`Cuxmix`, please refer to [Cuxmix](../../../ppcls/configs/ImageNet/DataAugment/ResNet50_Cutmix.yaml#L63-L65). +- The training accuracy (Acc) metric cannot be calculated when using `Mixup` or `Cutmix` for training, so you need to remove the `Metric.Train.TopkAcc` field in the configuration file, please refer to [Metric.Train.TopkAcc](../../../ppcls/configs/ImageNet/DataAugment/ResNet50_Cutmix.yaml#L125-L128). + +#### Q2.1.9: What are the fields `Global.pretrain_model` and `Global.checkpoints` used for in the training configuration file yaml? + +**A**： + +- When `fine-tune` is required, the path of the file of pre-training model weights can be configured via the field `Global.pretrain_model`, which usually has the suffix `.pdparams`. +- During training, the training program automatically saves the breakpoint information at the end of each epoch, including the optimizer information `.pdopt` and model weights information `.pdparams`. In the event that the training process is unexpectedly interrupted and needs to be resumed, the breakpoint information file saved during training can be configured via the field `Global.checkpoints`, for example by configuring `checkpoints: . /output/ResNet18/epoch_18` to restore the breakpoint information at the end of 18 epoch training. PaddleClas will automatically load `epoch_18.pdopt` and `epoch_18.pdparams` to continue training from 19 epoch. + + + +### 2.2 Image Classification + +#### Q2.2.1 How to distill the small model after pre-training the large model on 500M data and then distill the fintune model on 1M data in SSLD? + +**A**：The steps are as follows: + +1. Obtain the `ResNet50-vd` model based on the distillation of the open-source `ResNeXt101-32x16d-wsl` model from Facebook. +2. Use this `ResNet50-vd` to distill `MobilNetV3` on a 500W dataset. +3. Considering that the distribution of the 500W dataset is not exactly the same as that of the 100W data, this piece of data is finetuned on the 100W data to slightly improve the accuracy. + +#### Q2.2.2 nan appears in loss when training SwinTransformer + +**A**：When training SwinTransformer, please use `Paddle` `2.1.1` or above, and load the pre-trained model we provide. Also, the learning rate should be kept at an appropriate level. + + + +### 2.3 General Detection + +#### Q2.3.1 Why are there some images that are detected as the original image? + +**A**：The mainbody detection model returns the detection frame, but in fact, in order to make the subsequent recognition model more accurate, the original image is also returned along with the detection frame. Subsequently, the original image or the detection frame will be sorted according to its similarity with the images in the library, and the label of the image in the library with the highest similarity will be the label of the recognized image. + +#### Q2.3.2： + +**A**：A real-time detection presents high requirements for the detection speed; PP-YOLO is a lightweight target detection model provided by Paddle team, which strikes a good balance of detection speed and accuracy, you can try PP-YOLO for detection. For the use of PP-YOLO, you can refer to [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/configs/ppyolo/README_cn.md). + +#### Q2.3.3: For unknown labels, adding gallery dataset can be used for subsequent classification recognition (without training), but if the previous detection model cannot locate and detect the unknown labels, is it still necessary to train the previous detection model? + +**A**：If the detection model does not perform well on your own dataset, you need to finetune it again on your own detection dataset. + + + +### 2.4 Image Recognition + +#### Q2.4.1: Why is `Illegal instruction` reported during the recognition inference? + +**A**：If you are using the release/2.2 branch, it is recommended to update it to the release/2.3 branch, where we replaced the Möbius search model with the faiss search module, as described in [Vector Search Tutorial](../image_recognition_pipeline/vector_search_en.md). If you still have problems, you can contact us in the WeChat group or raise an issue on GitHub. + +#### Q2.4.2: How can recognition models be fine-tuned to train on the basis of pre-trained models? + +**A**：The fine-tuning training of the recognition model is similar to that of the classification model. The recognition model can be loaded with a pre-trained model of the product, and the training process can be found in [recognition model training](../../models_training/recognition_en.md), and we will continue to refine the documentation. + +#### Q2.4.3: Why does it fail to run all mini-batches in each epoch when training metric learning? + +**A**：When training metric learning, the Sampler used is DistributedRandomIdentitySampler, which does not sample all the images, resulting in each epoch sampling only part of the data, so it is normal that the mini-batch cannot run through the display. This issue has been optimized in the release/2.3 branch, please update to release/2.3 to use it. + +#### Q2.4.4: Why do some images have no recognition results? + +**A**：In the configuration file (e.g. inference_product.yaml), `IndexProcess.score_thres` controls the minimum value of cosine similarity of the recognized image to the image in the library. When the cosine similarity is less than this value, the result will not be printed. You can adjust this value according to your actual data. + + + +### 2.5 Vector Search + +#### Q2.5.1: Why is the error `assert text_num >= 2` reported after adding an image to the index? + +**A**：Make sure that the image path and the image name in data_file.txt is separated by a single table instead of a space. + +#### Q2.5.2: Do I need to rebuild the index to add new base data? + +**A**：Starting from release/2.3 branch, we have replaced the Möbius search model with the faiss search module, which already supports the addition of base data without building the base library, as described in [Vector Search Tutorial](../image_recognition_pipeline/vector_search_en.md). + +#### Q2.5.3: How to deal with the reported error clang: error: unsupported option '-fopenmp' when recompiling index.so in Mac? + +**A**： + +If you are using the release/2.2 branch, it is recommended to update it to the release/2.3 branch, where we replaced the Möbius search model with the faiss search module, as described in [Vector Search Tutorial](../image_recognition_pipeline/vector_search_en.md). If you still have problems, you can contact us in the user WeChat group or raise an issue on GitHub. + +#### Q2.5.4: How to set the parameter `pq_size` when build searches the base library? + +**A**： + +`pq_size` is a parameter of the PQ search algorithm, which can be simply understood as a "tiered" search algorithm. And `pq_size` is the "capacity" of each tier, so the setting of this parameter will affect the performance. However, in the case that the total data volume of the base library is not too large (less than 10,000), this parameter has little impact on the performance. So for most application scenarios, there is no need to modify this parameter when building the base library. For more details on the PQ search algorithm, see the related [paper](https://lear.inrialpes.fr/pubs/2011/JDS11/jegou_searching_with_quantization.pdf). + + + +### 2.6 Model Inference Deployment + +#### Q2.6.1: How to add the parameter of a module that is enabled by hub serving? + +**A**：See [hub serving parameters](../../../deploy/hubserving/clas/params.py) for more details. + +#### Q2.6.2: Why is the result not accurate enough when exporting the inference model for inference deployment? + +**A**: + +This problem is usually caused by the incorrect loading of the model parameters when exporting. First check the export log for something like the following. + +``` +UserWarning: Skip loading for ***. *** is not found in the provided dict. +``` + +If it exists, the model weights were not loaded successfully. Please further check the `Global.pretrained_model` field in the configuration file to see if the path of the model weights file is correctly configured. The suffix of the model weights file is usually `pdparams`, note that the file suffix is not required when configuring this path. + +#### Q2.6.3: How to convert the model to `ONNX` format? + +**A**： + +Paddle supports two ways and relies on the `paddle2onnx` tool, which first requires the installation of `paddle2onnx`. + +``` +pip install paddle2onnx +``` + +- From inference model to ONNX format model: + + Take the `combined` format inference model (containing `.pdmodel` and `.pdiparams` files) exported from the dynamic graph as an example, run the following command to convert the model format: + + ``` + paddle2onnx --model_dir ${model_path} --model_filename ${model_path}/inference.pdmodel --params_filename ${model_path}/inference.pdiparams --save_file ${save_path}/model.onnx --enable_onnx_checker True + ``` + + In the above commands： + + - `model_dir`: this parameter needs to contain `.pdmodel` and `.pdiparams` files. + - `model_filename`: this parameter is used to specify the path of the `.pdmodel` file under the parameter `model_dir`. + - `params_filename`: this parameter is used to specify the path of the `.pdiparams` file under the parameter `model_dir`. + - `save_file`: this parameter is used to specify the path to the directory where the converted model is saved. + + For the conversion of a non-`combined` format inference model exported from a static diagram (usually containing the file `__model__` and multiple parameter files), and more parameter descriptions, please refer to the official documentation of [paddle2onnx](https://github.com/PaddlePaddle/Paddle2ONNX/blob/develop/README.md#parameters). + +- Exporting ONNX format models directly from the model networking code. + + Take the model networking code of dynamic graphs as an example, the model class is a subclass that inherits from `paddle.nn.Layer` and the code is shown below: + + ```python + import paddle + from paddle.static import InputSpec + + class SimpleNet(paddle.nn.Layer): + def __init__(self): + pass + def forward(self, x): + pass + + net = SimpleNet() + x_spec = InputSpec(shape=[None, 3, 224, 224], dtype='float32', name='x') + paddle.onnx.export(layer=net, path="./SimpleNet", input_spec=[x_spec]) + ``` + + Among them： + + - `InputSpec()` function is used to describe the signature information of the model input, including the `shape`, `type` and `name` of the input data (can be omitted). + - The `paddle.onnx.export()` function needs to specify the model grouping object `net`, the save path of the exported model `save_path`, and the description of the model's input data `input_spec`. + + Note that the `paddlepaddle` `2.0.0` or above should be adopted.See [paddle.onnx.export](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/onnx/) for more details on the parameters of the `paddle.onnx.export()` function. diff --git a/src/PaddleClas/docs/en/faq_series/faq_selected_30_en.md b/src/PaddleClas/docs/en/faq_series/faq_selected_30_en.md new file mode 100644 index 0000000..54f6ca1 --- /dev/null +++ b/src/PaddleClas/docs/en/faq_series/faq_selected_30_en.md @@ -0,0 +1,291 @@ +# FAQ + +## Before You Read + +- We collect some frequently asked questions in issues and user groups since PaddleClas is open-sourced and provide brief answers, aiming to give some reference for the majority to save you from twists and turns. +- There are many talents in the field of image classification, recognition and retrieval with quickly updated models and papers, and the answers here mainly rely on our limited project practice, so it is not possible to cover all facets. We sincerely hope that the man of insight will help to supplement and correct the content, thanks a lot. + +## Catalogue + +- [1. 30 Questions About Image Classification](#1) + - [1.1 Basic Knowledge](#1.1) + - [1.2 Model Training](#1.2) + - [1.3 Data](#1.3) + - [1.4 Model Inference and Prediction](#1.4) +- [2. Application of PaddleClas](#2) + + +## 1. 30 Questions About Image Classification + + +### 1.1 Basic Knowledge + +- Q: How many classification metrics are commonly used in the field of image classification? +- A: + - For a single-label image classification (containing only 1 category and background), the evaluation metrics are Accuracy, Precision, Recall, F-score, etc. If TP(True Positive) means predicting positive class as positive, FP(False Positive) means predicting negative class as positive, TN( True Negative) means the negative class is predicted to be negative, and FN(False Negative) means the positive class is predicted to be negative. Then Accuracy=(TP + TN) / NUM, Precision=TP /(TP + FP), Recall=TP /(TP + FN). + - For the image classification problem with the number of classes greater than 1, the evaluation metrics are Accuary and Class-wise Accuracy. Accuary indicates the percentage of the number of images correctly predicted by all classes to the total number of images; Class-wise Accuracy is obtained by calculating the Accuracy for each class of images and then averaging the Accuracy of all classes. + +> > + +- Q: 怎样根据自己的任务选择合适的模型进行训练？How to choose the right training model? +- A: If you want to deploy on the server with a high requirement for accuracy but not model storage size or prediction speed, then it is recommended to use ResNet_vd, Res2Net_vd, DenseNet, Xception, etc., which are suitable for server-side models. If you want to deploy on the mobile side, then it is recommended to use MobileNetV3 and GhostNet. Meanwhile, we suggest you refer to the speed-accuracy metrics chart in [Model Library](../models/models_intro_en.md) when choosing models. + +> > + +- Q: How to initialize the parameters and what kind of initialization can speed up the convergence of the model? +- A: It is well known that the initialization of parameters can affect the final performance of the model. In general, if the target dataset is not very large, it is recommended to use the pre-trained model obtained by training ImageNet-1k for initialization. If the network is designed manually or there are no pre-trained weights based on ImageNet-1k training, you can use Xavier initialization or MSRA initialization, where the former is proposed for Sigmoid function, which is less friendly to RELU function. The deeper the network is, the smaller the variance of each layer input, the harder the network is to train. So when more RELU activation functions are used in the neural network, MSRA initialization is a better choice. + +> > + +- Q: What are the better solutions to the problem of parameter redundancy in deep neural networks? +- A: There are several major approaches to compressing models and reducing the model parameter redundancy, such as pruning, quantization, and knowledge distillation. Model pruning refers to removing relatively unimportant weights from the weight matrix and then fine-tuning the network again. Model quantization refers to a technique that converts floating-point computation into low-ratio specific-point computation, such as 8-bit, 4-bit, etc., which can effectively reduce the computational intensity, parameter size, and memory consumption of the model. Knowledge distillation refers to the use of a teacher model to guide a student model to learn a specific task, ensuring that the small model has a great performance improvement or even obtains similar accuracy metrics as the large model with the same number of parameters. + +> > + +- Q: How to choose the right classification model as a backbone network in other tasks, such as target detection, image segmentation, key point detection, etc.? + +- A: + + Without considering the speed, it is most recommended to use pre-training models and backbone networks with higher accuracy. A series of SSLD knowledge distillation pre-training models are open-sourced in PaddleClas, such as ResNet50_vd_ssld, Res2Net200_vd_26w_4s_ssld, etc., which excel in both model accuracy and speed. For specific tasks, such as image segmentation or key point detection, which require higher image resolution, it is recommended to use neural network models such as HRNet that can take into account both network depth and resolution. And PaddleClas also provides HRNet SSLD distillation series pre-training models including HRNet_W18_C_ssld, HRNet_W48_C_ssld, etc., which have very high accuracy. You can use these models and the backbone network to improve your own model accuracy on other tasks. + +> > + +- Q: What is the attention mechanism? What are the common methods of it? +- A: The Attention Mechanism (AM) originated from the study of human vision. Using the mechanism on computer vision tasks can effectively capture the useful regions in the images and thus improve the overall network performance. Currently, the most commonly used ones are [SE block](https://arxiv.org/abs/1709.01507), [SK-block](https://arxiv.org/abs/1903.06586), [Non-local block](https://arxiv. org/abs/1711.07971), [GC block](https://arxiv.org/abs/1904.11492), [CBAM](https://arxiv.org/abs/1807.06521), etc. The core idea is to learn the importance of feature maps in different regions or different channels, so that the network can pay more attention to the regions of salience. + + + +### 1.2 Model Training + +> > + +- Q: What will happen if a model with 10 million classes is trained during the image classification with deep convolutional networks? +- A: Because of the large number of parameters in the FC layer, the memory/video memory/model storage usage will increase significantly; the model convergence speed will also be slower. In this case, it is recommended to add a layer of FC with a smaller dimension before the last FC layer, which can drastically reduce the storage size of the model. + +> > + +- Q: What are the possible reasons if the model converges poorly during the training process? +- A: There are several points that can be investigated: (1) The data annotation should be checked to ensure that there are no problems with the labeling of the training and validation sets. (2) Try to adjust the learning rate (initially by a factor of 10). A learning rate that is too large (training oscillation) or too small (slow convergence) may lead to poor convergence. (3) Huge amount of data and an overly small model may prevent it from learning all the features of the data. (4) See if normalization is used in the data preprocessing process. It may be slower without normalization operation. (5) If the amount of data is relatively small, you can try to load the pre-trained model based on ImageNet-1k dataset provided in PaddleClas, which can greatly improve the training convergence speed. (6) There is a long tail problem in the dataset, you can refer to the [solution to the long tail problem of data](#long_tail). + +> > + +- Q: How to choose the right optimizer when training image classification tasks? +- A: Since the emergence of deep learning, there has been a lot of research on optimizers, which aim to minimize the loss function to find the right weights for a given task. Currently, the main optimizers used in the industry are SGD, RMSProp, Adam, AdaDelt, etc. Among them, since the SGD optimizer with momentum is widely used in academia and industry (only for classification tasks), most of the models we published also adopt this optimizer to achieve gradient descent of the loss function. It has two disadvantages, one is the slow convergence speed, and the other is the reliance on experiences of the initial learning rate setting. However, if the initial learning rate is set properly with a sufficient number of iterations, the optimizer will also stand out among many other optimizers, obtaining higher accuracy on the validation set. Some optimizers with adaptive learning rates, such as Adam and RMSProp, tend to converge fast, but the final convergence accuracy will be slightly worse. If you pursue faster convergence speed, we recommend using these adaptive learning rate optimizers, and SGD optimizers with momentum for higher convergence accuracy. + +- Q: What are the current mainstream learning rate decay strategies? How to choose? +- A: The learning rate is the speed at which the hyperparameters of the network weights are adjusted by the gradient of the loss function. The lower the learning rate, the slower the loss function will change. While using a low learning rate ensures that no local minimal values are missed, it also means that it takes longer to converge, especially if trapped in a plateau region. Throughout the whole training process, we cannot adopt the same learning rate to update the weights, otherwise, the optimal point cannot be reached, So we need to adjust the learning rate during the training. In the initial stage of training, since the weights are in a random initialization state and the loss function decreases fast, a larger learning rate can be set. And in the later stage of training, since the weights are close to the optimal value, a larger learning rate cannot further find the optimal value, so a smaller learning rate needs is a better choice. As for the learning rate decay strategy, many researchers or practitioners use piecewise_decay (step_decay), which is a stepwise decay learning rate. In addition, there are also other methods proposed by researchers, such as polynomial_decay, exponential_ decay, cosine_decay, etc. Among them, cosine_decay requires no adjustment of hyperparameters and has higher robustness, thus emerging as the preferred learning rate decay method to improve model accuracy. The learning rates of cosine_decay and piecewise_decay are shown in the following figure. It is easy to observe that cosine_decay keeps a large learning rate throughout the training, so it is slow in convergence, but its final effect is better than peicewise_decay. + +![](../../images/models/lr_decay.jpeg) + +> > + +- Q: What is the Warmup strategy? Where is it applied? +- A: The warmup strategy, which, as the name implies, is a warm-up for the learning rate with no direct adoption of maximum learning rate at the beginning of training, but to train the network with a gradually increasing rate, and then decay the learning rate when it peaks. When training a neural network with a large batch_size, it is recommended to use the warmup strategy. Experiments show that warmup can steadily improve the accuracy of the model when the batch_size is large. For example, when training MobileNetV3, we set the epoch in warmup to 5 by default, i.e., first increase the learning rate from 0 to the maximum value with 5 epochs, and then conduct the corresponding decay of the learning rate. + +> > + +- Q: What is `batch size`？How to choose the appropriate `batch size` during training? +- A: `batch size` is an important hyperparameter in neural networks training, whose value determines how much data is fed into the neural network for training at a time. According to the paper [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://arxiv.org/abs/1706.02677), when the value of `batch size` is linearly related to the value of learning rate, the convergence accuracy is almost unaffected. When training ImageNet-1k data, most of the neural networks choose an initial learning rate of 0.1 and a `batch size` of 256. Therefore, depending on the actual model size and video memory, the learning rate can be set to 0.1*k and the batch_size to 256*k. This setting can also be used as the initial parameter to further adjust the learning rate parameter and obtain better performance in real tasks. + +> > + +- Q: What is weight_decay？How to choose？ +- A: Overfitting is a common term in machine learning, which is simply understood as a model that performs well on training data but less satisfactory on test data. In image classification, there is also the problem of overfitting, and many regularization methods are proposed to avoid it, among which weight_decay is one of the widely used ways. When using SGD optimizer, weight_decay is equivalent to adding L2 regularization after the final loss function, which makes the weights of the network tend to choose smaller values, so eventually, the parameter values in the whole network tend to be more towards 0, and the generalization performance of the model is improved accordingly. In the implementation of major deep learning frameworks, this value means the coefficient before the L2 regularization, which is called L2Decay in the PaddlePaddle framework. The larger the coefficient is, the stronger the added regularization is, and the more the model tends to be underfitted. When training ImageNet, most networks set the value of this parameter to 1e-4, and in some smaller networks such as the MobileNet series network, the value is set between 1e-5 and 4e-5 to avoid the underfitting. Of course, the setting of this value is also related to specific datasets. When the dataset of the task is large, the network itself tends to be under-fitted and the value should be reduced appropriately, and when it is small, the network itself tends to be over-fitted and the value should be increased. The following table shows the accuracy of MobileNetV1_x0_25 on ImageNet-1k using different l2_decay. Since MobileNetV1_x0_25 is a relatively small network, too large a l2_decay will tend to underfit the network, so 3e-5 is a better choice in this network compared to 1e-4. + +| Model | L2_decay | Train acc1/acc5 | Test acc1/acc5 | +| ----------------- | -------- | --------------- | -------------- | +| MobileNetV1_x0_25 | 1e-4 | 43.79%/67.61% | 50.41%/74.70% | +| MobileNetV1_x0_25 | 3e-5 | 47.38%/70.83% | 51.45%/75.45% | + +> > + +- Q: What does label smoothing (label_smoothing) refer to? What is the effect? What kind of scenarios does it usually apply to? +- A: Label_smoothing is a regularization method in deep learning, whose full name is Label Smoothing Regularization (LSR). In the traditional classification task, the loss function is calculated by the cross-entropy of the real one hot label and the output of the neural network, while label_smoothing is a label smoothing of the real one hot label, so that the label learned by the network is no longer a hard label, but a soft label with a probability value, where the probability at the position corresponding to the category is the largest and others small. See the paper[2] for detailed calculation methods. In label_smoothing, the epsilon parameter describes the degree of label softening, the larger the value, the smaller the label probability value of the label vector after label smoothing, the smoother the label, and vice versa. The value is usually set to 0.1 in experiments training ImageNet-1k, and there is a steady increase in accuracy for models of the ResNet50 size and above after using label_smooting. The following table shows the accuracy metrics of ResNet50_vd before and after using label_smoothing. At the same time, since label_smoohing can be regarded as a regularization method, the accuracy improvement is not obvious or even decreases on a relatively small model. The following table shows the accuracy metrics of ResNet18 before and after using label_smoothing on ImageNet-1k. It is clear that the accuracy drops after using label_smoothing. + +| Model | Use_label_smoothing | Test acc1 | +| ----------- | ------------------- | --------- | +| ResNet50_vd | 0 | 77.9% | +| ResNet50_vd | 1 | 78.4% | +| ResNet18 | 0 | 71.0% | +| ResNet18 | 1 | 70.8% | + +> > + +- Q: How to determine the tuning strategy by the accuracy or loss of the training and validation sets during training? +- A: In the process of training a network, the accuracy of the training set and validation set are usually printed for each epoch, which portrays the performance of the model on both datasets. Generally speaking, it is good to have a comparable accuracy or a slightly higher accuracy in the training set than in the validation set. If we find that the accuracy of the training set is much higher than the validation set, it means that the training set is overfitted and we need to add more regularity, such as increasing the value of L2Decay, adding more data augmentation strategies, introducing label_smoothing strategies, etc. If we find that the accuracy of the training set is lower than the validation set, it means that the training set is probably underfitted, and the regularization effect should be weakened during the training, such as reducing the value of L2Decay, decreasing the data augmentation methods, increasing the area of the crop area, weakening the image stretching, removing label_smoothing, etc. + +> > + +- Q: How to improve the accuracy of my own dataset by pre-training the model? +- A: At this stage, it has become a common practice in the image recognition field to load pre-trained models to train their own tasks, which can often improve the accuracy of a particular task compared to training from random initialization. In general, the pre-training model widely used in the industry is obtained by training the ImageNet-1k dataset of 1.28 million images of 1000 classes. The fc layer weights of this pre-training model are a matrix of k*1000, where k is the number of neurons before the fc layer, and it is not necessary to load the fc layer weights when loading the pre-training weights. In terms of the learning rate, if your dataset is particularly small (e.g., less than 1,000), we recommend you to adopt a small initial learning rate, e.g., 0.001 (batch_size:256, the same below), so as not to corrupt the pre-training weights with a larger learning rate. If your training dataset is relatively large (>100,000), we suggest you try a larger initial learning rate, such as 0.01 or above. + + +### 1.3 Data + +> > + +- Q: What are the general steps involved in the data pre-processing for image classification? +- A: When training ResNet50 on ImageNet-1k dataset, an image is fed into the network, and there are several steps: image decoding, random cropping, random horizontal flipping, normalization, data rearrangement, group batching and feeding into the network. Image decoding refers to reading the image file into memory; random cropping refers to randomly stretching and cropping the read image to an image with the length and width of 224 ; random horizontal flipping refers to flipping the cropped image horizontally with a probability of 0.5; normalization refers to centering the data of each channel of the image by de-meaning, so that the data conforms to the `N(0,1)` normal distribution as much as possible; data rearrangement refers to changing the data from `[224,224,3]` format to `[3,224,224]`; and group batching refers to forming a batch of multiple images and feeding them into the network for training. + +> > + +- Q: How does random-crop affect the performance of small model training? +- A: In the standard preprocessing of ImageNet-1k data, the random_crop function defines two values, scale and ratio, which respectively determine the size of the image crop and the degree of image stretching, where the default value of the former is 0.08-1 (lower_scale-upper_scale), and the latter is 3/4-4/3 (lower_ratio-upper_ratio). In very small networks, this kind of data augmentation can lead to network underfitting and decreased accuracy. To the end, the data augmentation can be made weaker by increasing the crop area of the image or decreasing the stretching of the image. Weaker image transformation can be achieved by increasing the value of lower_scale or reducing the difference between lower_ratio and upper_scale, respectively. The following table shows the accuracy of training MobileNetV2_x0_25 with different lower_scale, and we can see that the training accuracy and verification accuracy are improved by increasing the crop area of the images. + +| Model | Range of Scale | Train_acc1/acc5 | Test_acc1/acc5 | +| ----------------- | -------------- | --------------- | -------------- | +| MobileNetV2_x0_25 | [0.08,1] | 50.36%/72.98% | 52.35%/75.65% | +| MobileNetV2_x0_25 | [0.2,1] | 54.39%/77.08% | 53.18%/76.14% | + +> > + +- Q: What are the common data augmentation methods currently available to increase the richness of training samples when the amount of data is insufficient? +- A: PaddleClas classifies data augmentation methods into three categories, which are image transformation, image cropping and image aliasing. Image transformation mainly includes AutoAugment and RandAugment, image cropping contains CutOut, RandErasing, HideAndSeek and GridMask, and image aliasing comprises Mixup and Cutmix. More detailed introduction to data augmentation can be found in the chapter of [Data Augmentation ](../algorithm_introduction/DataAugmentation_en.md). + +> > + +- Q: For image classification scenarios where occlusion is common, what data augmentation methods should be used to improve the accuracy of the model? +- A: During the training, you can try to adopt cropping data augmentations including CutOut, RandErasing, HideAndSeek and GridMask on the training set, so that the model can learn not only the significant regions but also the non-significant regions, thus better performing the recognition task. + +> > + +- Q: What data augmentation methods should be used to improve model accuracy in the case of complex color transformations? +- A: Consider using the data augmentation strategies of AutoAugment or RandAugment, both of which include rich color transformations such as sharpening and histogram equalization, allowing the model to be more robust to these transformations during the training process. + +> > + +- Q: How do Mixup and Cutmix work? Why are they effective methods of data augmentation? +- A: Mixup generates a new image by linearly overlaying two images, and the corresponding labels also undertake the same process for training, while Cutmix crops a random region of interest (ROI) from an image and overlays the corresponding region in the current image, and the labels are linearly overlaid in proportion to the image area. They actually generate different samples and labels from the training set and for the learning of the network, thus enriching the samples. + +> > + +- Q: What is the size of the training dataset for an image classification task that does not require high accuracy? +- A: The amount of the training data is related to the complexity of the problem to be solved. The greater the difficulty and the higher the accuracy requirement, the larger the dataset needs to be. And it is a universal rule that the more training data the better the result in practice. Of course, in general, 10-20 images per category with pre-trained models can guarantee the basic classification effect; or at least 100-200 images per category without pre-training models. + + + +> > + +- Q: What are the common methods currently used for datasets with long-tailed distributions? +- A:(1) the categories with fewer data can be resampled to increase the probability of their occurrence; (2) the loss can be modified to increase the loss weight of images in categories corresponding to fewer images; (3) the method of transfer learning can be borrowed to learn generic knowledge from common categories and then migrate to the categories with fewer samples. + + + +### 1.4 Model Inference and Prediction + +> > + +- Q: How to deal with the poor recognition performance when the original image is taken for classification with only a small part of the image being the foreground object of interest? +- A: A mainbody detection model can be added before classification to detect the foreground objects, which can greatly improve the final recognition results. If time cost is not a concern, multi-crop can also be used to fuse all the predictions to determine the final category. + +> > + +- Q: What are the currently recommended model inference methods? +- A: After the model is trained, it is recommended to use the exported inference model to make inferences based on the Paddle inference engine, which currently supports python inference and cpp inference. If you want to deploy the inference model based on service, it is recommended to use the PaddleServing. + +> > + +- Q: What are the appropriate inference methods to further improve the model accuracy after training? +- A:(1) A larger inference scale can be used, e.g., 224 for training, then 288 or 320 for inference, which will directly bring about a 0.5% improvement in accuracy. (2) Test Time Augmentation (TTA) can be used to create multiple copies of the test set by rotating, flipping, color transforming, and so on, and then fuse all the prediction results, which can greatly improve its accuracy and robustness. (3) Of course, a multi-model fusion strategy can also be adopted to fuse the prediction results of multiple models for the same images. + +> > + +- Q: How to choose the best model for the fusion of multiple models? +- A: Without considering the inference speed, models with the highest possible are recommended; it is also suggested to choose models with different structures or series for fusion. For example, the model fusion results of ResNet50_vd and Xception65 tend to be better than those of ResNet50_vd and ResNet101_vd with similar accuracy. + +> > + +- Q: What are the common acceleration methods when using a fixed model for inference? +- A: (1) Using a GPU with better performance; (2) increasing the batch size; (3) using TenorRT and FP16 half-precision floating-point methods. + + + +## 2. Application of PaddleClas + +> > + +- Q: Why can't I import parameters even though I have specified the address of the folder where the pre-trained model is located during evaluation and inference? +- A: When loading a pretrained model, you need to specify the prefix of it. For example, if the folder where the pretrained model parameters are located is `output/ResNet50_vd/19` and the name of the pretrained model parameters is `output/ResNet50_vd/19/ppcls.pdparams`, then the `pretrained_model ` parameter needs to be specified as `output/ResNet50_vd/19/ppcls`, and PaddleClas will automatically complete the `.pdparams` suffix. + +> > + +- Q: Why the final accuracy is always about 0.3% lower than the official one when evaluating the `EfficientNetB0_small` model? +- A: The `EfficientNet` series network uses `cubic interpolation` for resize (the interpolation value of the resize parameter is set to 2), while other models are None by default, so the interpolation value of resize needs to be explicitly specified during training and evaluation. Specifically, you can refer to the ResizeImage parameter in the preprocessing process in the following configuration. + +``` + Eval: + dataset: + name: ImageNetDataset + image_root: ./dataset/ILSVRC2012/ + cls_label_path: ./dataset/ILSVRC2012/val_list.txt + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - ResizeImage: + resize_short: 256 + interpolation: 2 + - CropImage: + size: 224 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + +> > + +- Q: Why `TypeError: __init__() missing 1 required positional argument: 'sync_cycle'` is reported when using visualdl under python2？ +- A: Currently visualdl only supports running under python3 with a required version of 2.0 or higher. If visualdl is not the right version, you can install it as follows: `pip3 install visualdl -i https://mirror.baidu.com/pypi/simple` + +> > + +- Q: Why is it that the inference speed of a single image by ResNet50_vd is much lower than the benchmark provided by the official website while the CPU is much faster than GPU? +- A: The model inference needs to be initialized, and it is time-consuming. Therefore, when counting the inference speed, we need to run a batch of images, remove the inference time of the first few images, and then count the average time.GPU is slower than CPU to test a single image because the initialization of GPU is much slower than CPU. + +> > + +- Q: Can grayscale maps be used for model training? +- A: The grayscale image can also be used for model training, but the input shape of the model needs to be modified to `[1, 224, 224]`, and the data augmentation also needs to be adapted. However, for better use of the PaddleClas code, it is recommended to adapt the grayscale image to a 3-channel image for training (RGB channels have equal pixel values). + +> > + +- Q: How to train the model on windows or cpu? +- A: You can refer to [Getting Started Tutorial](../models_training/classification_en.md) for detailed tutorials on model training, evaluation and inference in Linux , Windows, CPU, and other environments. + +> > + +- Q: How to use label smoothing in model training? +- A: This can be set in the `Loss` field in the configuration file as follows. `epsilon=0.1` means set the value to 0.1, if the `epsilon` field is not set, then `label smoothing` will not be used. + +``` +Loss: + Train: + - CELoss: + weight: 1.0 + epsilon: 0.1 +``` + +> > + +- Q: Is the 10W class image classification pre-training model provided by PaddleClas available for model inference? +- A: This 10W class image classification pre-training model does not provide parameters for the fc fully connected layer, which cannot be used for model inference but is available for model fine-tuning at present. + +> > + +- Q: Why is `Error: Pass tensorrt_subgraph_pass has not been registered` reported When using `deploy/python/predict_cls.py` for model prediction? +- A: If you want to use TensorRT for model prediction and inference, you need to install or compile PaddlePaddle with TensorRT by yourself. For Linux, Windows, macOS users, you can refer to [download inference library](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html). If there is no required version, you need to compile and install it locally, which is detailed in [source code compilation](https://paddleinference.paddlepaddle.org.cn/user_guides/source_compile.html). + +> > + +- Q: How to train with Automatic Mixed Precision (AMP) during training? +- A: You can refer to [ResNet50_fp16.yaml](../../../ppcls/configs/ImageNet/ResNet/ResNet50_fp16.yaml). Specifically, if you want your configuration file to support automatic mixed precision during model training, you can add the following information to the file. + +``` +# mixed precision training +AMP: + scale_loss: 128.0 + use_dynamic_loss_scaling: True + use_pure_fp16: &use_pure_fp16 True +``` diff --git a/src/PaddleClas/docs/en/faq_series/index.rst b/src/PaddleClas/docs/en/faq_series/index.rst new file mode 100644 index 0000000..69a9f2d --- /dev/null +++ b/src/PaddleClas/docs/en/faq_series/index.rst @@ -0,0 +1,10 @@ +faq_series +================================ + +.. toctree:: + :maxdepth: 2 + + faq_2021_s2_en.md + faq_2021_s1_en.md + faq_2020_s1_en.md + faq_selected_30_en.md diff --git a/src/PaddleClas/docs/en/image_recognition_pipeline/feature_extraction_en.md b/src/PaddleClas/docs/en/image_recognition_pipeline/feature_extraction_en.md new file mode 100644 index 0000000..f86562a --- /dev/null +++ b/src/PaddleClas/docs/en/image_recognition_pipeline/feature_extraction_en.md @@ -0,0 +1,182 @@ +# Feature Extraction + +## Catalogue + +- [1.Introduction](#1) +- [2.Network Structure](#2) +- [3.General Recognition Models](#3) +- [4.Customized Feature Extraction](#4) + - [4.1 Data Preparation](#4.1) + - [4.2 Model Training](#4.2) + - [4.3 Model Evaluation](#4.3) + - [4.4 Model Inference](#4.4) + + +## 1.Introduction + +Feature extraction plays a key role in image recognition, which serves to transform the input image into a fixed dimensional feature vector for subsequent [vector search](./vector_search_en.md). Good features boast great similarity preservation, i.e., in the feature space, pairs of images with high similarity should have higher feature similarity (closer together), and pairs of images with low similarity should have less feature similarity (further apart). [Deep Metric Learning](../algorithm_introduction/metric_learning_en.md) is applied to explore how to obtain features with high representational power through deep learning. + + +## 2.Network Structure + +In order to customize the image recognition task flexibly, the whole network is divided into Backbone, Neck, Head, and Loss. The figure below illustrates the overall structure: + +![img](../../images/feature_extraction_framework_en.png) + +Functions of the above modules : + +- **Backbone**: Specifies the backbone network to be used. It is worth noting that the ImageNet-based pre-training model provided by PaddleClas has an output of 1000 for the last layer, which demands for customization according to the required feature dimensions. +- **Neck**: Used for feature augmentation and feature dimension transformation. Here it can be a simple Linear Layer for feature dimension transformation, or a more complex FPN structure for feature augmentation. +- **Head**: Used to transform features into logits. In addition to the common Fc Layer, cosmargin, arcmargin, circlemargin and other modules are all available choices. +- **Loss**: Specifies the Loss function to be used. It is designed as a combined form to facilitate the combination of Classification Loss and Pair_wise Loss. + + +## 3.General Recognition Models + +In PP-Shitu, we have [PP_LCNet_x2_5](../models/PP-LCNet.md) as the backbone network, Linear Layer for Neck, [ArcMargin](../../../ppcls/arch/gears/arcmargin.py) for Head, and CELoss for Loss. See the details in [General Recognition_configuration files](../../../ppcls/configs/GeneralRecognition/). The involved training data covers the following seven public datasets: + +| Datasets | Data Size | Class Number | Scenarios | URL | +| ------------ | --------- | ------------ | ------------------ | ------------------------------------------------------------ | +| Aliproduct | 2498771 | 50030 | Commodities | [URL](https://retailvisionworkshop.github.io/recognition_challenge_2020/) | +| GLDv2 | 1580470 | 81313 | Landmarks | [URL](https://github.com/cvdfoundation/google-landmark) | +| VeRI-Wild | 277797 | 30671 | Vehicle | [URL](https://github.com/PKU-IMRE/VERI-Wild) | +| LogoDet-3K | 155427 | 3000 | Logo | [URL](https://github.com/Wangjing1551/LogoDet-3K-Dataset) | +| iCartoonFace | 389678 | 5013 | Cartoon Characters | [URL](http://challenge.ai.iqiyi.com/detail?raceId=5def69ace9fcf68aef76a75d) | +| SOP | 59551 | 11318 | Commodities | [URL](https://cvgl.stanford.edu/projects/lifted_struct/) | +| Inshop | 25882 | 3997 | Commodities | [URL](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html) | +| **Total** | **5M** | **185K** | ---- | ---- | + +The results are shown in the table below: + +| Model | Aliproduct | VeRI-Wild | LogoDet-3K | iCartoonFace | SOP | Inshop | Latency(ms) | +| ------------- | ---------- | --------- | ---------- | ------------ | ----- | ------ | ----------- | +| PP-LCNet-2.5x | 0.839 | 0.888 | 0.861 | 0.841 | 0.793 | 0.892 | 5.0 | + +- Evaluation metric: `Recall@1` +- CPU of the speed evaluation machine: `Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz`. +- Evaluation conditions for the speed metric: MKLDNN enabled, number of threads set to 10 +- Address of the pre-training model: [General recognition pre-training model](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/general_PPLCNet_x2_5_pretrained_v1.0.pdparams) + + +## 4.Customized Feature Extraction + +Customized feature extraction refers to retraining the feature extraction model based on one's own task. It consists of four main steps: 1) data preparation, 2) model training, 3) model evaluation, and 4) model inference. + + +### 4.1 Data Preparation + +To start with, customize your dataset based on the task (See [Format description](../data_preparation/recognition_dataset_en.md#1) for the dataset format). Before initiating the model training, modify the data-related content in the configuration files, including the address of the dataset and the class number. The corresponding locations in configuration files are shown below: + +``` + Head: + name: ArcMargin + embedding_size: 512 + class_num: 185341 #Number of class +``` + +``` +Train: + dataset: + name: ImageNetDataset + image_root: ./dataset/ #The directory where the train dataset is located + cls_label_path: ./dataset/train_reg_all_data.txt #The address of label file for train dataset +``` + +``` + Query: + dataset: + name: VeriWild + image_root: ./dataset/Aliproduct/. #The directory where the query dataset is located + cls_label_path: ./dataset/Aliproduct/val_list.txt. #The address of label file for query dataset +``` + +``` + Gallery: + dataset: + name: VeriWild + image_root: ./dataset/Aliproduct/ #The directory where the gallery dataset is located + cls_label_path: ./dataset/Aliproduct/val_list.txt. #The address of label file for gallery dataset +``` + + +### 4.2 Model Training + +- Single machine single card training + +``` +export CUDA_VISIBLE_DEVICES=0 +python tools/train.py -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml +``` + +- Single machine multi card training + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" tools/train.py \ + -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml +``` + +**Note:** The configuration file adopts `online evaluation` by default, if you want to speed up the training and remove `online evaluation`, just add `-o eval_during_train=False` after the above command. After training, the final model files `latest`, `best_model` and the training log file `train.log` will be generated under the directory output. Among them, `best_model` is utilized to store the best model under the current evaluation metrics while`latest` is adopted to store the latest generated model, making it convenient to resume the training from where it was interrupted. + +- Resumption of Training： + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" tools/train.py \ + -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ + -o Global.checkpoint="output/RecModel/latest" +``` + + +### 4.3 Model Evaluation + +- Single Card Evaluation + +``` +export CUDA_VISIBLE_DEVICES=0 +python tools/eval.py \ +-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ +-o Global.pretrained_model="output/RecModel/best_model" +``` + +- Multi Card Evaluation + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" tools/eval.py \ + -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ + -o Global.pretrained_model="output/RecModel/best_model" +``` + +**Recommendation:** It is suggested to employ multi-card evaluation, which can quickly obtain the feature set of the overall dataset using multi-card parallel computing, accelerating the evaluation process. + + +### 4.4 Model Inference + +Two steps are included in the inference: 1)exporting the inference model; 2)obtaining the feature vector. + +#### 4.4.1 Export Inference Model + +``` +python tools/export_model.py \ +-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ +-o Global.pretrained_model="output/RecModel/best_model" +``` + +The generated inference models are under the directory `inference`, which comprises three files, namely, `inference.pdmodel`、`inference.pdiparams`、`inference.pdiparams.info`. Among them, `inference.pdmodel` serves to store the structure of inference model while `inference.pdiparams` and `inference.pdiparams.info` are mobilized to store model-related parameters. + +#### 4.4.2 Obtain Feature Vector + +``` +cd deploy +python python/predict_rec.py \ +-c configs/inference_rec.yaml \ +-o Global.rec_inference_model_dir="../inference" +``` + +The output format of the obtained features is shown in the figure below:![img](../../images/feature_extraction_output.png) + +In practical use, however, business operations require more than simply obtaining features. To further perform image recognition by feature retrieval, please refer to the document [vector search](./vector_search_en.md). diff --git a/src/PaddleClas/docs/en/image_recognition_pipeline/index.rst b/src/PaddleClas/docs/en/image_recognition_pipeline/index.rst new file mode 100644 index 0000000..95aa67d --- /dev/null +++ b/src/PaddleClas/docs/en/image_recognition_pipeline/index.rst @@ -0,0 +1,9 @@ +image_recognition_pipeline +================================ + +.. toctree:: + :maxdepth: 2 + + mainbody_detection_en.md + feature_extraction_en.md + vector_search_en.md diff --git a/src/PaddleClas/docs/en/image_recognition_pipeline/mainbody_detection_en.md b/src/PaddleClas/docs/en/image_recognition_pipeline/mainbody_detection_en.md new file mode 100644 index 0000000..9ff310e --- /dev/null +++ b/src/PaddleClas/docs/en/image_recognition_pipeline/mainbody_detection_en.md @@ -0,0 +1,242 @@ +# Mainbody Detection + +The mainbody detection technology is currently a widely used detection technology, which refers to a whole image recognition process of identifying the coordinate position of one or more objects and then cropping down the corresponding area for recognition. Mainbody detection is the first step of the recognition task, which can effectively improve the recognition accuracy. + +This tutorial will introduce the technology from three aspects, namely, the datasets, model selection and model training. + +## Catalogue + +- [1. Dataset](#1) +- [2. Model Selection](#2) + - [2.1 Lightweight Mainbody Detection Model](#2.1) + - [2.2 Server-side Mainbody Detection Model](#2.2) +- [3. Model Training](#3) + - [3.1 Prepare For the Environment](#3.1) + - [3.2 Prepare For the Dataset](#3.2) + - [3.3 Configuration Files](#3.3) + - [3.4 Begin the Training Process](#3.4) + - [3.5 Model Prediction](#3.5) + - [3.6 Model Export and Inference Deployment](#3.6) + + + +## 1. Dataset + +The datasets we used for mainbody detection tasks are shown in the following table. + +| Dataset | Image Number | Image Number Used in Mainbody Detection | Scenarios | Dataset Link | +| ------------ | ------------ | --------------------------------------- | ----------------- | ---------------------------------------------------------- | +| Objects365 | 170W | 6k | General Scenarios | [Link](https://www.objects365.org/overview.html) | +| COCO2017 | 12W | 5k | General Scenarios | [Link](https://cocodataset.org/) | +| iCartoonFace | 2k | 2k | Cartoon Face | [Link](https://github.com/luxiangju-PersonAI/iCartoonFace) | +| LogoDet-3k | 3k | 2k | Logo | [Link](https://github.com/Wangjing1551/LogoDet-3K-Dataset) | +| RPC | 3k | 3k | Product | [Link](https://rpc-dataset.github.io/) | + +In the actual training process, all datasets are mixed together. Categories of all the labeled boxes are modified as `foreground`, and the detection model we trained only contains one category (`foreground`). + + + +## 2. Model Selection + +There are a wide variety of object detection methods, such as the commonly used two-stage detectors (FasterRCNN series, etc.), single-stage detectors (YOLO, SSD, etc.), anchor-free detectors (FCOS, etc.) and so on. PaddleDetection has its self-developed PP-YOLO models for server-side scenarios and PicoDet models for end-side scenarios (CPU and mobile), which all take the lead in the area. + +Build on the studies above, PaddleClas provides lightweight and server-side main body detection models for end-side scenarios and server-side scenarios respectively. The table below presents the average mAP of the 5 datasets and the comparison of their model sizes and inference speed. + +| Model | Model Structure | Download Link of Pre-trained Model | Download Link of Inference Model | mAP | Size of Inference Model (MB) | Inference Time per Image (preprocessing excluded)(ms) | +| ------------------------------------ | --------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ----- | ---------------------------- | ----------------------------------------------------- | +| Lightweight Mainbody Detection Model | PicoDet | [Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_pretrained.pdparams) | [Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar) | 40.1% | 30.1 | 29.8 | +| Server-side Mainbody Detection Model | PP-YOLOv2 | [Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/ppyolov2_r50vd_dcn_mainbody_v1.0_pretrained.pdparams) | [Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar) | 42.5% | 210.5 | 466.6 | + +Notes: + +- Detailed information of the CPU of the speed evaluation machine：`Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz`.The speed indicator is the testing result when mkldnn is on and the number of threads is set to 10. +- Mainbody detection has a time-consuming preprocessing procedure, with an average time of about 40 to 55 ms per image in the above machine. Therefore, it is not included in the inference time. + + + +### 2.1 Lightweight Mainbody Detection Model + +PicoDet, introduced by [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection), is an object detection algorithm applied to CPU or mobile-side scenarios. It integrates the following optimization algorithm. + +- [ATSS](https://arxiv.org/abs/1912.02424) +- [Generalized Focal Loss](https://arxiv.org/abs/2006.04388) +- Cosine learning rate decay +- Cycle-EMA +- Lightweight detection head + +For more details of optimized PicoDet and benchmark, you can refer to [Tutorials of PicoDet Models](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/picodet/README.md). + +To balance the detection speed and effects in lightweight mainbody detection tasks, we adopt PPLCNet_x2_5 as the backbone of the model and revise the image scale for training and inference to 640x640, with the rest configured the same as [picodet_m_shufflenetv2_416_coco.yml](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/picodet/picodet_m_shufflenetv2_416_coco.yml). The final detection model is obtained after the training of customized mainbody detection datasets. + + + +### 2.2 Server-side Mainbody Detection Model + +PP-YOLO is proposed by [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection). It greatly optimizes the yolov3 model from multiple perspectives such as backbone, data augmentation, regularization strategy, loss function, and post-processing. It reaches the state of the art in terms of "speed-precision". The optimization strategy is as follows. + +- Better backbone: ResNet50vd-DCN +- Larger training batch size of 8 GPUs and mini-batch size of 24 on each GPU, which is corresponding to learning rate and the number of iterations. +- [Drop Block](https://arxiv.org/abs/1810.12890) +- [Exponential Moving Average](https://www.investopedia.com/terms/e/ema.asp) +- [IoU Loss](https://arxiv.org/pdf/1902.09630.pdf) +- [Grid Sensitive](https://arxiv.org/abs/2004.10934) +- [Matrix NMS](https://arxiv.org/pdf/2003.10152.pdf) +- [CoordConv](https://arxiv.org/abs/1807.03247) +- [Spatial Pyramid Pooling](https://arxiv.org/abs/1406.4729) +- Better Pre-trained Model + +For more information about PP-YOLO, you can refer to [PP-YOLO tutorial](https://github.com/PaddlePaddle/PaddleDetection/blob/release%2F2.1/configs/ppyolo/README.md). + +In the mainbody detection task, we use `ResNet50vd-DCN` as our backbone for better performance. The config file is [ppyolov2_r50vd_dcn_365e_coco.yml](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml), in which the dataset path is modified to the customized mainbody detection dataset. The final detection model can be downloaded [here](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar). + + + +## 3. Model Training + +This section mainly talks about how to train your own mainbody detection model using PaddleDetection on your own datasets. + + + +### 3.1 Prepare For the Environment + +Download PaddleDetection and install requirements. + +```shell +cd +git clone https://github.com/PaddlePaddle/PaddleDetection.git + +cd PaddleDetection +# install requirements +pip install -r requirements.txt +``` + +For more installation tutorials, please refer to [Installation Tutorial](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/docs/tutorials/INSTALL.md) + + + +### 3.2 Prepare For the Dataset + +For customized dataset, you should convert it to COCO format. Please refer to [Customized Dataset Tutorial](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/static/docs/tutorials/Custom_DataSet.md) to build your own datasets with COCO format. + +In mainbody detection task, all the objects belong to foregroud. Therefore, `category_id` of all the objects in the annotation file should be modified to 1. And the `categories` map should be modified as follows, in which just class `foregroud` is included. + +``` +[{u'id': 1, u'name': u'foreground', u'supercategory': u'foreground'}] +``` + + + +### 3.3 Configuration Files + +We use `configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml` to train the model, mode details are as follows. + + [![img](../../images/det/PaddleDetection_config.png)](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/images/det/PaddleDetection_config.png) + + + +`ppyolov2_r50vd_dcn_365e_coco.yml` depends on other configuration files, their meanings are as follows. + +``` +coco_detection.yml：path of train/eval/test dataset. + +runtime.yml：public runtime parameters, including whethre to use GPU, epoch number for checkpoint saving, etc. + +optimizer_365e.yml：learning rate and optimizer. + +ppyolov2_r50vd_dcn.yml：model architecture and backbone. + +ppyolov2_reader.yml：train/eval/test reader, such as batch size, the number of concurrently loaded sub-processes, etc., and includes post-read pre-processing operations, such as resize, data enhancement, etc. +``` + +In mainbody detection task, you need to modify `num_classes` in `datasets/coco_detection.yml` to 1 (only `foreground` is included), while modify the paths of the training and testing datasets to those of the customized datasets. + +In addition, the above files can also be modified according to real situations, for example, if the video memory is overflowing, the batch size and learning rate can be reduced in equal proportion. + + + +### 3.4 Begin the Training Process + +PaddleDetection supports many ways of training process. + +- Training using single GPU + +``` +# not needed for windows and Mac +export CUDA_VISIBLE_DEVICES=0 +python tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml +``` + +- Training using multiple GPUs + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --eval +``` + +--eval: evaluation while training + +- (**Recommend**) Model finetune If you want to finetune the trained model in PaddleClas on your own datasets, you can run the following command. + +``` +export CUDA_VISIBLE_DEVICES=0 +# assign pretrain_weights, load the general mainbody-detection pretrained model +python tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml -o pretrain_weights=https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/ppyolov2_r50vd_dcn_mainbody_v1.0_pretrained.pdparams +``` + +- Resume training + + you can use `-r` to load checkpoints and resume training. + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --eval -r output/ppyolov2_r50vd_dcn_365e_coco/10000 +``` + +Note: If `Out of memory error` occurs, you can try to decrease `batch_size` in `ppyolov2_reader.yml` while reducing learning rate in equal proportion. + + + +### 3.5 Model Prediction + +Use the following command to finish the prediction process. + +``` +export CUDA_VISIBLE_DEVICES=0 +python tools/infer.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --infer_img=your_image_path.jpg --output_dir=infer_output/ --draw_threshold=0.5 -o weights=output/ppyolov2_r50vd_dcn_365e_coco/model_final +``` + +`--draw_threshold` is an optional parameter. According to NMS calculation, different thresholds will produce different results. `keep_top_k` indicates the maximum number of output targets, with a default value of 100 that can be modified according to their actual situation. + + + +### 3.6 Model Export and Inference Deployment + +Use the following to export the inference model： + +``` +python tools/export_model.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --output_dir=./inference -o weights=output/ppyolov2_r50vd_dcn_365e_coco/model_final.pdparams +``` + +The inference model will be saved under the directory `inference/ppyolov2_r50vd_dcn_365e_coco`, which contains`infer_cfg.yml` (optional for mainbody detection), `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel`. + +Note： Inference model that `PaddleDetection` exports is named `model.xxx`，if you want to keep it consistent with PaddleClas，you can rename `model.xxx` to `inference.xxx` for subsequent inference deployment of mainbody detection. + +For more model export tutorials, please refer to [EXPORT_MODEL](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/deploy/EXPORT_MODEL.md). + +The final directory contains `inference/ppyolov2_r50vd_dcn_365e_coco`, `inference.pdiparams`, `inference.pdiparams.info`, and `inference.pdmodel`，among which`inference.pdiparams` refers to saved weight files of the inference model while `inference.pdmodel` stands for structural files. + +After exporting the model, the path of the detection model can be changed to the inference model path to complete the prediction task. + +Take product recognition as an example，you can modify the field `Global.det_inference_model_dir` in its config file [inference_product.yaml](../../../deploy/configs/inference_product.yaml) to the directory of exported inference model, and then finish the detection and recognition of the product with reference to [Quick Start for Image Recognition](../quick_start/quick_start_recognition_en.md). + +## FAQ + +#### Q: Is it compatible with other mainbody detection models？ + +- A: Yes, but the current preprocessing process only supports PicoDet and YOLO models, so it is recommended to use these two for training. If you want to use other models such as Faster RCNN, you need to revise the logic of preprocessing in accordance with that of PaddleDetection. You are welcomed to resort to Github Issue or WeChat group for any needs or questions. + +#### Q: Can I modify the prediction scale of mainbody detection? + +- A: Yes, but there are 2 things that require attention + - The mainbody detection model provided in PaddleClas is trained based on `640x640` resolution, so this is also the default value of prediction process. The accuracy will be reduced if other resolutions are used. + - When exporting the model, it is recommended to modify the resolution of the exported model to keep it consistent with the prediction process. diff --git a/src/PaddleClas/docs/en/image_recognition_pipeline/vector_search_en.md b/src/PaddleClas/docs/en/image_recognition_pipeline/vector_search_en.md new file mode 100644 index 0000000..9a6d545 --- /dev/null +++ b/src/PaddleClas/docs/en/image_recognition_pipeline/vector_search_en.md @@ -0,0 +1,120 @@ +# Vector Search + +Vector search finds wide applications in image recognition and image retrieval. It aims to obtain the similarity ranking for a given query vector by performing a similarity or distance calculation of feature vectors with all the vectors to be queried in an established vector library. In the image recognition system, [Faiss](https://github.com/facebookresearch/faiss) is adopted for corresponding support, please check [the official website of Faiss](https://github.com/facebookresearch/faiss) for more details. The main advantages of `Faiss` can be generalized as the following: + +- Great adaptability: support Windows, Linux, and MacOS systems +- Easy installation: support `python` interface and direct installation with `pip` +- Rich algorithms: support a variety of search algorithms to cover different scenarios +- Support both CPU and GPU, which accelerates the search process + +It is worth noting that the current version of `PaddleClas` **only uses CPU for vector retrieval** for the moment in pursuit of better adaptability. + +[![img](../../images/structure.jpg)](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/images/structure.jpg) + +As shown in the figure above, two parts constitute the vector search in the whole `PP-ShiTu` system. + +- The green part: the establishment of search libraries for the search query, while providing functions such as adding and deleting images. +- The blue part: the search function, i.e., given the feature vector of a picture and return the label of similar images in the library. + +This document mainly introduces the installation of the search module in PaddleClas, the adopted search algorithms, the library building process, and the parameters in the relevant configuration files. + +------ + +## Catalogue + +- [1. Installation of the Search Library](#1) +- [2. Search Algorithms](#2) +- [3. Introduction of and Configuration Files](#3) + - [3.1 Parameters of Library Building and Configuration Files](#3.1) + - [3.2 Parameters of Search Configuration Files](#3.2) + + + +## 1. Installation of the Search Library + +`Faiss` can be installed as follows: + +``` +pip install faiss-cpu==1.7.1post2 +``` + +If the above cannot be properly used, please `uninstall` and then `install` again, especially when you are using`windows`. + + + +## 2. Search Algorithms + +Currently, the search module in `PaddleClas` supports the following three search algorithms: + +- **HNSW32**: A graph indexing method boasts high retrieval accuracy and fast speed. However, the feature library only supports the function of adding images, not deleting image features. (Default method) +- **IVF**: An inverted index search method with fast speed but slightly lower precision. The feature library supports functions of adding and deleting image features. +- **FLAT**: A violent search algorithm presenting the highest precision, but slower retrieval speed in face of large data volume. The feature library supports functions of adding and deleting image features. + +Each search algorithm can find its right place in different scenarios. `HNSW32`, as the default method, strikes a balance between accuracy and speed, see its detailed introduction in the [official document](https://github.com/facebookresearch/faiss/wiki). + + + +## 3. Introduction of Configuration Files + +Configuration files involving the search module are under `deploy/configs/`, where `build_*.yaml` is related to building the feature library, and `inference_*.yaml` is the inference file for retrieval or classification. + + + +### 3.1 Parameters of Library Building and Configuration Files + +The building of the library is detailed as follows: + +``` +# Enter deploy directory +cd deploy +# Change the yaml file to the specific one you need +python python/build_gallery.py -c configs/build_***.yaml +``` + +The `yaml` file is configured as follows for library building, please make necessary corrections to fit the real operation. The construction will extract the features of the images under `image_root` according to the image list in `data_file` and store them under `index_dir` for subsequent search. + +The `data_file` stores the path and label of the image file, with each line presenting the format `image_path label`. The intervals are spaced by the `delimiter` parameter in the `yaml` file. + +The specific model parameters for feature extraction can be found in the `yaml` file. + +``` +# indexing engine config +IndexProcess: + index_method: "HNSW32" # supported: HNSW32, IVF, Flat + index_dir: "./recognition_demo_data_v1.1/gallery_product/index" + image_root: "./recognition_demo_data_v1.1/gallery_product/" + data_file: "./recognition_demo_data_v1.1/gallery_product/data_file.txt" + index_operation: "new" # suported: "append", "remove", "new" + delimiter: "\t" + dist_type: "IP" + embedding_size: 512 +``` + +- **index_method**: the search algorithm. It currently supports three, HNSW32, IVF, and Flat. +- **index_dir**: the folder where the built feature library is stored. +- **image_root**: the location of the folder where the annotated images needed to build the feature library are stored. +- **data_file**: the data list of the annotated images needed to build the feature library, the format of each line: relative_path label. +- **index_operation**: the operation to build a library: `new` for initiating an operation, `append` for adding the image feature of data_file to the feature library, `remove` for deleting the image of data_file from the feature library. +- **delimiter**: delimiter for each line in **data_file** +- **dist_type**: the method of similarity calculation adopted in feature matching. For example, Inner Product(`IP`) and Euclidean distance(`L2`). +- **embedding_size**: feature dimensionality + + + +### 3.2 Parameters of Search Configuration Files + +To integrate the search into the overall `PP-ShiTu` process, please refer to `The Introduction of PP-ShiTu Image Recognition System` in [README](../../../README_en.md). Please check the [Quick Start for Image Recognition](../quick_start/quick_start_recognition_en.md) for the specific operation of the search. + +The search part is configured as follows. Please refer to `deploy/configs/inference_*.yaml` for the complete version. + +``` +IndexProcess: + index_dir: "./recognition_demo_data_v1.1/gallery_logo/index/" + return_k: 5 + score_thres: 0.5 +``` + +The following are new parameters other than those of the library building configuration file: + +- `return_k`: `k` results are returned +- `score_thres`: the threshold for retrieval and match diff --git a/src/PaddleClas/docs/en/index.rst b/src/PaddleClas/docs/en/index.rst new file mode 100644 index 0000000..0bb50d3 --- /dev/null +++ b/src/PaddleClas/docs/en/index.rst @@ -0,0 +1,18 @@ +欢迎使用PaddleClas图像分类库！ +================================ + +.. toctree:: + :maxdepth: 2 + + models_training/index + introduction/index + image_recognition_pipeline/index + others/index + faq_series/index + data_preparation/index + installation/index + models/index + advanced_tutorials/index + algorithm_introduction/index + inference_deployment/index + quick_start/index diff --git a/src/PaddleClas/docs/en/inference_deployment/cpp_deploy_en.md b/src/PaddleClas/docs/en/inference_deployment/cpp_deploy_en.md new file mode 100644 index 0000000..3f92b66 --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/cpp_deploy_en.md @@ -0,0 +1,298 @@ +# Server-side C++ inference + +This tutorial will introduce the detailed steps of deploying the PaddleClas classification model on the server side. The deployment of the recognition model will be supported in the near future. Please look forward to it. + +--- + +## Catalogue + +- [1. Prepare the environment](#1) + - [1.1 Compile OpenCV](#1.1) + - [1.2 Compile or download the Paddle Inference Library](#1.2) + - [1.2.1 Compile from the source code](#1.2.1) + - [1.2.2 Direct download and installation](#1.2.2) +- [2. Compile](#2) + - [2.1 Compile PaddleClas C++ inference demo](#2.1) + - [2.2 Compile config lib and cls lib](#2.2) +- [3. Run](#3) + - [3.1 Prepare inference model](#3.1) + - [3.2 Run demo](#3.2) + + +## 1. Prepare the environment + +### Environment + +- Linux, docker is recommended. +- Windows, compilation based on `Visual Studio 2019 Community` is supported. In addition, you can refer to [How to use PaddleDetection to make a complete project](https://zhuanlan.zhihu.com/p/145446681) to compile by generating the `sln solution`. +- This document mainly introduces the compilation and inference of PaddleClas using C++ in Linux environment. + + +### 1.1 Compile opencv + +* First of all, you need to download the source code compiled package in the Linux environment from the opencv official website. Taking opencv3.4.7 as an example, the download and uncompress command are as follows. + +``` +wget https://github.com/opencv/opencv/archive/3.4.7.tar.gz +tar -xf 3.4.7.tar.gz +``` + +Finally, you can see the folder of `opencv-3.4.7/` in the current directory. + +* Compile opencv, the opencv source path (`root_path`) and installation path (`install_path`) should be set by yourself. Among them, `root_path` is the downloaded opencv source code path, and `install_path` is the installation path of opencv. In this case, the opencv source is `./opencv-3.4.7`. + +```shell +cd ./opencv-3.4.7 +export root_path=$PWD +export install_path=${root_path}/opencv3 +``` + +* After entering the opencv source code path, you can compile it in the following way. + + +```shell +rm -rf build +mkdir build +cd build + +cmake .. \ + -DCMAKE_INSTALL_PREFIX=${install_path} \ + -DCMAKE_BUILD_TYPE=Release \ + -DBUILD_SHARED_LIBS=OFF \ + -DWITH_IPP=OFF \ + -DBUILD_IPP_IW=OFF \ + -DWITH_LAPACK=OFF \ + -DWITH_EIGEN=OFF \ + -DCMAKE_INSTALL_LIBDIR=lib64 \ + -DWITH_ZLIB=ON \ + -DBUILD_ZLIB=ON \ + -DWITH_JPEG=ON \ + -DBUILD_JPEG=ON \ + -DWITH_PNG=ON \ + -DBUILD_PNG=ON \ + -DWITH_TIFF=ON \ + -DBUILD_TIFF=ON + +make -j +make install +``` + +* After `make install` is completed, the opencv header file and library file will be generated in this folder for later PaddleClas source code compilation. + +Take opencv3.4.7 for example, the final file structure under the opencv installation path is as follows. **NOTICE**:The following file structure may be different for different Versions of Opencv. + +``` +opencv3/ +|-- bin +|-- include +|-- lib64 +|-- share +``` + + +### 1.2 Compile or download the Paddle Inference Library + +* There are 2 ways to obtain the Paddle Inference Library, described in detail below. + + +#### 1.2.1 Compile from the source code +* If you want to get the latest Paddle Inference Library features, you can download the latest code from Paddle GitHub repository and compile the inference library from the source code. +* You can refer to [Paddle Inference Library](https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/05_inference_deployment/inference/build_and_install_lib_en.html#build-from-source-code) to get the Paddle source code from github, and then compile To generate the latest inference library. The method of using git to access the code is as follows. + + +```shell +git clone https://github.com/PaddlePaddle/Paddle.git +``` + +* After entering the Paddle directory, the compilation method is as follows. + +```shell +rm -rf build +mkdir build +cd build + +cmake .. \ + -DWITH_CONTRIB=OFF \ + -DWITH_MKL=ON \ + -DWITH_MKLDNN=ON \ + -DWITH_TESTING=OFF \ + -DCMAKE_BUILD_TYPE=Release \ + -DWITH_INFERENCE_API_TEST=OFF \ + -DON_INFER=ON \ + -DWITH_PYTHON=ON +make -j +make inference_lib_dist +``` + +For more compilation parameter options, please refer to the official website of the Paddle C++ inference library:[https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/05_inference_deployment/inference/build_and_install_lib_en.html#build-from-source-code](https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/05_inference_deployment/inference/build_and_install_lib_en.html#build-from-source-code). + + +* After the compilation process, you can see the following files in the folder of `build/paddle_inference_install_dir/`. + +``` +build/paddle_inference_install_dir/ +|-- CMakeCache.txt +|-- paddle +|-- third_party +|-- version.txt +``` + +Among them, `paddle` is the Paddle library required for C++ prediction later, and `version.txt` contains the version information of the current inference library. + + +#### 1.2.2 Direct download and installation + +* Different cuda versions of the Linux inference library (based on GCC 4.8.2) are provided on the +[Paddle Inference Library official website](https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/05_inference_deployment/inference/build_and_install_lib_en.html). You can view and select the appropriate version of the inference library on the official website. + +* Please select the `develop` version. + +* After downloading, use the following method to uncompress. + +``` +tar -xf paddle_inference.tgz +``` + +Finally you can see the following files in the folder of `paddle_inference/`. + + + +## 2. Compile + + +### 2.1 Compile PaddleClas C++ inference demo + + +* The compilation commands are as follows. The addresses of Paddle C++ inference library, opencv and other Dependencies need to be replaced with the actual addresses on your own machines. + +```shell +sh tools/build.sh +``` + +Specifically, the content in `tools/build.sh` is as follows. + +```shell +OPENCV_DIR=your_opencv_dir +LIB_DIR=your_paddle_inference_dir +CUDA_LIB_DIR=your_cuda_lib_dir +CUDNN_LIB_DIR=your_cudnn_lib_dir +TENSORRT_DIR=your_tensorrt_lib_dir + +BUILD_DIR=build +rm -rf ${BUILD_DIR} +mkdir ${BUILD_DIR} +cd ${BUILD_DIR} +cmake .. \ + -DPADDLE_LIB=${LIB_DIR} \ + -DWITH_MKL=ON \ + -DDEMO_NAME=clas_system \ + -DWITH_GPU=OFF \ + -DWITH_STATIC_LIB=OFF \ + -DWITH_TENSORRT=OFF \ + -DTENSORRT_DIR=${TENSORRT_DIR} \ + -DOPENCV_DIR=${OPENCV_DIR} \ + -DCUDNN_LIB=${CUDNN_LIB_DIR} \ + -DCUDA_LIB=${CUDA_LIB_DIR} \ + +make -j +``` + +In the above parameters of command: + +* `OPENCV_DIR` is the opencv installation path; + +* `LIB_DIR` is the download (`paddle_inference` folder) or the generated Paddle Inference Library path (`build/paddle_inference_install_dir` folder); + +* `CUDA_LIB_DIR` is the cuda library file path, in docker; it is `/usr/local/cuda/lib64`; + +* `CUDNN_LIB_DIR` is the cudnn library file path, in docker it is `/usr/lib/x86_64-linux-gnu/`. + +* `TENSORRT_DIR` is the tensorrt library file path，in dokcer it is `/usr/local/TensorRT6-cuda10.0-cudnn7/`，TensorRT is just enabled for GPU. + +After the compilation is completed, an executable file named `clas_system` will be generated in the `build` folder. + + +### 2.2 Compile config lib and cls lib + +In addition to compiling the demo directly, you can also compile only config lib and cls lib by running the following command: + +```shell +sh tools/build_lib.sh +``` + +The contents of the above command are as follows: + +```shell +OpenCV_DIR=path/to/opencv +PADDLE_LIB_DIR=path/to/paddle + +BUILD_DIR=./lib/build +rm -rf ${BUILD_DIR} +mkdir ${BUILD_DIR} +cd ${BUILD_DIR} +cmake .. \ + -DOpenCV_DIR=${OpenCV_DIR} \ + -DPADDLE_LIB=${PADDLE_LIB_DIR} \ + -DCMP_STATIC=ON \ + +make +``` + +The specific description of each compilation option is as follows: +* `DOpenCV_DIR`: The directory to the OpenCV compilation library. In this example, it is `opencv-3.4.7/opencv3/share/OpenCV`. Note that there needs to be a `OpenCVConfig.cmake` file under this directory; +* `DPADDLE_LIB`: The directory to the paddle prediction library which generally is the path of `paddle_inference` downloaded and decompressed or compiled, such as `build/paddle_inference_install_dir`. Note that there should be two subdirectories `paddle` and `third_party` in this directory; +* `DCMP_STATIC`: Whether to compile config lib and cls lib into static link library (`.a`). The default is `ON`. If you need to compile into dynamic link library (`.so`), please set it to `OFF`. + +After executing the above commands, the dynamic link libraries (`libcls.so` and `libconfig.so`) or static link libraries (`cls.a` and `libconfig.a`) of config lib and cls lib will be generated in the directory. In the [2.1 Compile PaddleClas C++ inference demo](#2.1), you can specify the compilation option `DCLS_LIB` and `DCONFIG_LIB` to the path of the existing link library of `cls lib` and `config lib`, which can also be used for development. + + + +## 3. Run the demo + + +### 3.1 Prepare the inference model + +* You can refer to [Model inference](../../../tools/export_model.py)，export the inference model. After the model is exported, assuming it is placed in the `inference` directory, the directory structure is as follows. + +``` +inference/ +|--cls_infer.pdmodel +|--cls_infer.pdiparams +``` + +**NOTICE**: Among them, `cls_infer.pdmodel` file stores the model structure information and the `cls_infer.pdiparams` file stores the model parameter information.The paths of the two files need to correspond to the parameters of `cls_model_path` and `cls_params_path` in the configuration file `tools/config.txt`. + + +### 3.2 Run demo + +First, please modify the `tools/config.txt` and `tools/run.sh`. + +* Some key words in `tools/config.txt` is as follows. + * use_gpu: Whether to use GPU. + * gpu_id: GPU id. + * gpu_mem：GPU memory. + * cpu_math_library_num_threads：Number of thread for math library acceleration. + * use_mkldnn：Whether to use mkldnn. + * use_tensorrt: Whether to use tensorRT. + * use_fp16：Whether to use Float16 (half precision), it is just enabled when use_tensorrt is set as 1. + * cls_model_path: Model path of inference model. + * cls_params_path: Params path of inference model. + * resize_short_size：Short side length of the image after resize. + * crop_size：Image size after center crop. + +* You could modify `tools/run.sh`(`./build/clas_system ./tools/config.txt ./docs/imgs/ILSVRC2012_val_00000666.JPEG`): + * ./build/clas_system: the path of executable file compiled; + * ./tools/config.txt: the path of config; + * ./docs/imgs/ILSVRC2012_val_00000666.JPEG: the path of image file to be predicted. + +* Then execute the following command to complete the classification of an image. + +```shell +sh tools/run.sh +``` + +* The prediction results will be shown on the screen, which is as follows. + +![](../../images/inference_deployment/cpp_infer_result.png) + +* In the above results,`class id` represents the id corresponding to the category with the highest confidence, and `score` represents the probability that the image belongs to that category. diff --git a/src/PaddleClas/docs/en/inference_deployment/export_model_en.md b/src/PaddleClas/docs/en/inference_deployment/export_model_en.md new file mode 100644 index 0000000..8fe3c46 --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/export_model_en.md @@ -0,0 +1,103 @@ +# Export model + +PaddlePaddle supports exporting inference model for deployment. Compared with training, inference model files store network weights and network structures persistently, and PaddlePaddle supports more fast prediction engine loading inference model to deployment. + +--- + +## Catalogue + +- [1. Environmental preparation](#1) +- [2. Export classification model](#2) +- [3. Export mainbody detection model](#3) +- [4. Export recognition model](#4) +- [5. Parameter description](#5) + + + + +## 1. Environmental preparation + +First, refer to the [Installing PaddlePaddle](../installation/install_paddle_en.md) and the [Installing PaddleClas](../installation/install_paddleclas_en.md) to prepare environment. + + + +## 2. Export classification model + +Change the working directory to PaddleClas: + +```shell +cd /path/to/PaddleClas +``` + +Taking the classification model ResNet50_vd as an example, download the pre-trained model: + +```shell +wget -P ./cls_pretrain/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_pretrained.pdparams +``` + +The above model weights is trained by ResNet50_vd model on ImageNet1k dataset and training configuration file is `ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml`. To export the inference model, just run the following command: + +```shell +python tools/export_model.py + -c ./ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml \ + -o Global.pretrained_model=./cls_pretrain/ResNet50_vd_pretrained \ + -o Global.save_inference_dir=./deploy/models/class_ResNet50_vd_ImageNet_infer +``` + + + +## 3. Export mainbody detection model + +About exporting mainbody detection model in details, please refer[mainbody detection](../image_recognition_pipeline/mainbody_detection_en.md). + + + +## 4. Export recognition model + +Change the working directory to PaddleClas: + +```shell +cd /path/to/PaddleClas +``` + +Take the feature extraction model in products recognition as an example, download the pretrained model: + +```shell +wget -P ./product_pretrain/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/product_ResNet50_vd_Aliproduct_v1.0_pretrained.pdparams +``` + +The above model weights file is trained by ResNet50_vd on AliProduct dataset, and the training configuration file is `ppcls/configs/Products/ResNet50_vd_Aliproduct.yaml`. The command to export inference model is as follow: + +```shell +python3 tools/export_model.py \ + -c ./ppcls/configs/Products/ResNet50_vd_Aliproduct.yaml \ + -o Global.pretrained_model=./product_pretrain/product_ResNet50_vd_Aliproduct_v1.0_pretrained \ + -o Global.save_inference_dir=./deploy/models/product_ResNet50_vd_aliproduct_v1.0_infer +``` + +Notice, the inference model exported by above command is truncated on embedding layer, so the output of the model is n-dimensional embedding feature. + + + +## 5. Parameter description + +In the above model export command, the configuration file used must be the same as the training configuration file. The following fields in the configuration file are used to configure exporting model parameters. + +* `Global.image_shape`：To specify the input data size of the model, which does not contain the batch dimension; +* `Global.save_inference_dir`：To specify directory of saving inference model files exported; +* `Global.pretrained_model`：To specify the path of model weight file saved during training. This path does not need to contain the suffix `.pdparams` of model weight file; + +The exporting model command will generate the following three files: + +* `inference.pdmodel`：To store model network structure information; +* `inference.pdiparams`：To store model network weight information; +* `inference.pdiparams.info`：To store the parameter information of the model, which can be ignored in the classification model and recognition model; + +The inference model exported is used to deployment by using prediction engine. You can refer the following docs according to different deployment modes / platforms + +* [Python inference](./python_deploy_en.md) +* [C++ inference](./cpp_deploy_en.md)(Only support classification) +* [Python Whl inference](./whl_deploy_en.md)(Only support classification) +* [PaddleHub Serving inference](./paddle_hub_serving_deploy_en.md)(Only support classification) +* [PaddleServing inference](./paddle_serving_deploy_en.md) +* [PaddleLite inference](./paddle_lite_deploy_en.md)(Only support classification) diff --git a/src/PaddleClas/docs/en/inference_deployment/index.rst b/src/PaddleClas/docs/en/inference_deployment/index.rst new file mode 100644 index 0000000..1e3c344 --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/index.rst @@ -0,0 +1,19 @@ +inference_deployment +================================ + +.. toctree:: + :maxdepth: 2 + + export_model_en.md + python_deploy_en.md + cpp_deploy_en.md + paddle_serving_deploy_en.md + paddle_hub_serving_deploy_en.md + paddle_lite_deploy_en.md + whl_deploy_en.md + + + + + + diff --git a/src/PaddleClas/docs/en/inference_deployment/paddle_hub_serving_deploy_en.md b/src/PaddleClas/docs/en/inference_deployment/paddle_hub_serving_deploy_en.md new file mode 100644 index 0000000..56e68fa --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/paddle_hub_serving_deploy_en.md @@ -0,0 +1,236 @@ +# Service deployment based on PaddleHub Serving + +PaddleClas supports rapid service deployment through Paddlehub. At present, it supports the deployment of image classification. Please look forward to the deployment of image recognition. + +--- + +## Catalogue + +- [1. Introduction](#1) +- [2. Prepare the environment](#2) +- [3. Download inference model](#3) +- [4. Install Service Module](#4) +- [5. Start service](#5) + - [5.1 Start with command line parameters](#5.1) + - [5.2 Start with configuration file](#5.2) +- [6. Send prediction requests](#6) +- [7. User defined service module modification](#7) + + +## 1. Introduction + +HubServing service pack contains 3 files, the directory is as follows: + +``` +hubserving/clas/ + └─ __init__.py Empty file, required + └─ config.json Configuration file, optional, passed in as a parameter when using configuration to start the service + └─ module.py Main module file, required, contains the complete logic of the service + └─ params.py Parameter file, required, including parameters such as model path, pre- and post-processing parameters +``` + + +## 2. Prepare the environment + +```shell +# Install version 2.0 of PaddleHub +pip3 install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple +``` + + +## 3. Download inference model + +Before installing the service module, you need to prepare the inference model and put it in the correct path. The default model path is: + +* Model structure file: `PaddleClas/inference/inference.pdmodel` +* Model parameters file: `PaddleClas/inference/inference.pdiparams` + +**Notice**: +* The model file path can be viewed and modified in `PaddleClas/deploy/hubserving/clas/params.py`. +* It should be noted that the prefix of model structure file and model parameters file must be `inference`. +* More models provided by PaddleClas can be obtained from the [model library](../models/models_intro_en.md). You can also use models trained by yourself. + + +## 4. Install Service Module + +* On Linux platform, the examples are as follows. +```shell +cd PaddleClas/deploy +hub install hubserving/clas/ +``` + +* On Windows platform, the examples are as follows. +```shell +cd PaddleClas\deploy +hub install hubserving\clas\ +``` + + +## 5. Start service + + +### 5.1 Start with command line parameters + +This method only supports CPU. Command as follow: + +```shell +$ hub serving start --modules Module1==Version1 \ + --port XXXX \ + --use_multiprocess \ + --workers \ +``` + +**parameters：** + +|parameters|usage| +|-|-| +|--modules/-m|PaddleHub Serving pre-installed model, listed in the form of multiple Module==Version key-value pairs
*`When Version is not specified, the latest version is selected by default`*| +|--port/-p|Service port, default is 8866| +|--use_multiprocess|Enable concurrent mode, the default is single-process mode, this mode is recommended for multi-core CPU machines
*`Windows operating system only supports single-process mode`*| +|--workers|The number of concurrent tasks specified in concurrent mode, the default is `2*cpu_count-1`, where `cpu_count` is the number of CPU cores| + +For example, start service: + +```shell +hub serving start -m clas_system +``` + +This completes the deployment of a service API, using the default port number 8866. + + +### 5.2 Start with configuration file + +This method supports CPU and GPU. Command as follow: + +```shell +hub serving start --config/-c config.json +``` + +Wherein, the format of `config.json` is as follows: + +```json +{ + "modules_info": { + "clas_system": { + "init_args": { + "version": "1.0.0", + "use_gpu": true, + "enable_mkldnn": false + }, + "predict_args": { + } + } + }, + "port": 8866, + "use_multiprocess": false, + "workers": 2 +} +``` + +- The configurable parameters in `init_args` are consistent with the `_initialize` function interface in `module.py`. Among them, + - when `use_gpu` is `true`, it means that the GPU is used to start the service. + - when `enable_mkldnn` is `true`, it means that use MKL-DNN to accelerate. +- The configurable parameters in `predict_args` are consistent with the `predict` function interface in `module.py`. + +**Note:** +- When using the configuration file to start the service, other parameters will be ignored. +- If you use GPU prediction (that is, `use_gpu` is set to `true`), you need to set the environment variable CUDA_VISIBLE_DEVICES before starting the service, such as: ```export CUDA_VISIBLE_DEVICES=0```, otherwise you do not need to set it. +- **`use_gpu` and `use_multiprocess` cannot be `true` at the same time.** +- **When both `use_gpu` and `enable_mkldnn` are set to `true` at the same time, GPU is used to run and `enable_mkldnn` will be ignored.** + +For example, use GPU card No. 3 to start the 2-stage series service: + +```shell +cd PaddleClas/deploy +export CUDA_VISIBLE_DEVICES=3 +hub serving start -c hubserving/clas/config.json +``` + + +## 6. Send prediction requests + +After the service starting, you can use the following command to send a prediction request to obtain the prediction result: + +```shell +cd PaddleClas/deploy +python hubserving/test_hubserving.py server_url image_path +``` + +Two required parameters need to be passed to the script: + +- **server_url**: service address，format of which is +`http://[ip_address]:[port]/predict/[module_name]` +- **image_path**: Test image path, can be a single image path or an image directory path +- **batch_size**: [**Optional**] batch_size. Default by `1`. +- **resize_short**: [**Optional**] In preprocessing, resize according to short size. Default by `256`。 +- **crop_size**: [**Optional**] In preprocessing, centor crop size. Default by `224`。 +- **normalize**: [**Optional**] In preprocessing, whether to do `normalize`. Default by `True`。 +- **to_chw**: [**Optional**] In preprocessing, whether to transpose to `CHW`. Default by `True`。 + +**Notice**: +If you want to use `Transformer series models`, such as `DeiT_***_384`, `ViT_***_384`, etc., please pay attention to the input size of model, and need to set `--resize_short=384`, `--crop_size=384`. + +**Eg.** +```shell +python hubserving/test_hubserving.py --server_url http://127.0.0.1:8866/predict/clas_system --image_file ./hubserving/ILSVRC2012_val_00006666.JPEG --batch_size 8 +``` + +The returned result is a list, including the `top_k`'s classification results, corresponding scores and the time cost of prediction, details as follows. + +``` +list: The returned results +└─ list: The result of first picture + └─ list: The top-k classification results, sorted in descending order of score + └─ list: The scores corresponding to the top-k classification results, sorted in descending order of score + └─ float: The time cost of predicting the picture, unit second +``` + +**Note：** If you need to add, delete or modify the returned fields, you can modify the corresponding module. For the details, refer to the user-defined modification service module in the next section. + + +## 7. User defined service module modification + +If you need to modify the service logic, the following steps are generally required: + +1. Stop service +```shell +hub serving stop --port/-p XXXX +``` + +2. Modify the code in the corresponding files, like `module.py` and `params.py`, according to the actual needs. You need re-install(hub install hubserving/clas/) and re-deploy after modifing `module.py`. +After modifying and installing and before deploying, you can use `python hubserving/clas/module.py` to test the installed service module. + +For example, if you need to replace the model used by the deployed service, you need to modify model path parameters `cfg.model_file` and `cfg.params_file` in `params.py`. Of course, other related parameters may need to be modified at the same time. Please modify and debug according to the actual situation. + +3. Uninstall old service module +```shell +hub uninstall clas_system +``` + +4. Install modified service module +```shell +hub install hubserving/clas/ +``` + +5. Restart service +```shell +hub serving start -m clas_system +``` + +**Note**: + +Common parameters can be modified in params.py: +* Directory of model files(include model structure file and model parameters file): + ```python + "inference_model_dir": + ``` +* The number of Top-k results returned during post-processing: + ```python + 'topk': + ``` +* Mapping file corresponding to label and class ID during post-processing: + ```python + 'class_id_map_file': + ``` + +In order to avoid unnecessary delay and be able to predict in batch, the preprocessing (include resize, crop and other) is completed in the client, so modify [test_hubserving.py](../../../deploy/hubserving/test_hubserving.py#L35-L52) if necessary. diff --git a/src/PaddleClas/docs/en/inference_deployment/paddle_lite_deploy_en.md b/src/PaddleClas/docs/en/inference_deployment/paddle_lite_deploy_en.md new file mode 100644 index 0000000..12d45dd --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/paddle_lite_deploy_en.md @@ -0,0 +1,270 @@ +# Tutorial of PaddleClas Mobile Deployment + +This tutorial will introduce how to use [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) to deploy PaddleClas models on mobile phones. + +Paddle-Lite is a lightweight inference engine for PaddlePaddle. It provides efficient inference capabilities for mobile phones and IoTs, and extensively integrates cross-platform hardware to provide lightweight deployment solutions for mobile-side deployment issues. + +If you only want to test speed, please refer to [The tutorial of Paddle-Lite mobile-side benchmark test](../extension/paddle_mobile_inference_en.md). + +--- + +## Catalogue + +- [1. Preparation](#1) + - [1.1 Build Paddle-Lite library](#1.1) + - [1.2 Download inference library for Android or iOS](#1.2) +- [2. Start running](#2) + - [2.1 Inference Model Optimization](#2.1) + - [2.1.1 [RECOMMEND] Use pip to install Paddle-Lite and optimize model](#2.1.1) + - [2.1.2 Compile Paddle-Lite to generate opt tool](#2.1.2) + - [2.1.3 Demo of get the optimized model](#2.1.3) + - [2.2 Run optimized model on Phone](#2.2) +- [3. FAQ](#3) + + +## 1. Preparation + +PaddeLite currently supports the following platforms: +- Computer (for compiling Paddle-Lite) +- Mobile phone (arm7 or arm8) + + +### 1.1 Prepare cross-compilation environment + +The cross-compilation environment is used to compile the C++ demos of Paddle-Lite and PaddleClas. + +For the detailed compilation directions of different development environments, please refer to the corresponding [document](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html). + + +## 1.2 Download inference library for Android or iOS + +|Platform|Inference Library Download Link| +|-|-| +|Android|[arm7](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/Android/gcc/inference_lite_lib.android.armv7.gcc.c++_static.with_extra.with_cv.tar.gz) / [arm8](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/Android/gcc/inference_lite_lib.android.armv8.gcc.c++_static.with_extra.with_cv.tar.gz)| +|iOS|[arm7](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/iOS/inference_lite_lib.ios.armv7.with_cv.with_extra.tiny_publish.tar.gz) / [arm8](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/iOS/inference_lite_lib.ios.armv8.with_cv.with_extra.tiny_publish.tar.gz)| + +**NOTE**: + +1. If you download the inference library from [Paddle-Lite official document](https://paddle-lite.readthedocs.io/zh/latest/quick_start/release_lib.html#android-toolchain-gcc), please choose `with_extra=ON` , `with_cv=ON` . + +2. It is recommended to build inference library using [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) develop branch if you want to deploy the [quantitative](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/deploy/slim/quantization/README_en.md) model to mobile phones. Please refer to the [link](https://paddle-lite.readthedocs.io/zh/latest/user_guides/Compile/Android.html#id2) for more detailed information about compiling. + + +The structure of the inference library is as follows: + +``` +inference_lite_lib.android.armv8/ +|-- cxx C++ inference library and header files +| |-- include C++ header files +| | |-- paddle_api.h +| | |-- paddle_image_preprocess.h +| | |-- paddle_lite_factory_helper.h +| | |-- paddle_place.h +| | |-- paddle_use_kernels.h +| | |-- paddle_use_ops.h +| | `-- paddle_use_passes.h +| `-- lib C++ inference library +| |-- libpaddle_api_light_bundled.a C++ static library +| `-- libpaddle_light_api_shared.so C++ dynamic library +|-- java Java inference library +| |-- jar +| | `-- PaddlePredictor.jar +| |-- so +| | `-- libpaddle_lite_jni.so +| `-- src +|-- demo C++ and java demos +| |-- cxx C++ demos +| `-- java Java demos +``` + + +## 2. Start running + + +## 2.1 Inference Model Optimization + +Paddle-Lite provides a variety of strategies to automatically optimize the original training model, including quantization, sub-graph fusion, hybrid scheduling, Kernel optimization and so on. In order to make the optimization process more convenient and easy to use, Paddle-Lite provides `opt` tool to automatically complete the optimization steps and output a lightweight, optimal executable model. + +**NOTE**: If you have already got the `.nb` file, you can skip this step. + + +### 2.1.1 [RECOMMEND] Use `pip` to install Paddle-Lite and optimize model + +* Use pip to install Paddle-Lite. The following command uses `pip3.7` . + +```shell +pip install paddlelite==2.8 +``` +**Note**：The version of `paddlelite`'s wheel must match that of inference lib. + +* Use `paddle_lite_opt` to optimize inference model, the parameters of `paddle_lite_opt` are as follows: + +| Parameters | Explanation | +| ----------------------- | ------------------------------------------------------------ | +| --model_dir | Path to the PaddlePaddle model (no-combined) file to be optimized. | +| --model_file | Path to the net structure file of PaddlePaddle model (combined) to be optimized. | +| --param_file | Path to the net weight files of PaddlePaddle model (combined) to be optimized. | +| --optimize_out_type | Type of output model, `protobuf` by default. Supports `protobuf` and `naive_buffer` . Compared with `protobuf`, you can use`naive_buffer` to get a more lightweight serialization/deserialization model. If you need to predict on the mobile-side, please set it to `naive_buffer`. | +| --optimize_out | Path to output model, not needed to add `.nb` suffix. | +| --valid_targets | The executable backend of the model, `arm` by default. Supports one or some of `x86` , `arm` , `opencl` , `npu` , `xpu`. If set more than one, please separate the options by space, and the `opt` tool will choose the best way automatically. If need to support Huawei NPU (DaVinci core carried by Kirin 810/990 SoC), please set it to `npu arm` . | +| --record_tailoring_info | Whether to enable `Cut the Library Files According To the Model` , `false` by default. If need to record kernel and OP infos of optimized model, please set it to `true`. | + +In addition, you can run `paddle_lite_opt` to get more detailed information about how to use. + + +### 2.1.2 Compile Paddle-Lite to generate `opt` tool + +Optimizing model requires Paddle-Lite's `opt` executable file, which can be obtained by compiling the Paddle-Lite. The steps are as follows: + +```shell +# get the Paddle-Lite source code, if have gotten , please skip +git clone https://github.com/PaddlePaddle/Paddle-Lite.git +cd Paddle-Lite +git checkout develop +# compile +./lite/tools/build.sh build_optimize_tool +``` + +After the compilation is complete, the `opt` file is located under `build.opt/lite/api/`. + +`opt` tool is used in the same way as `paddle_lite_opt` , please refer to [2.1.1](#2.1.1). + + +### 2.1.3 Demo of get the optimized model + +Taking the `MobileNetV3_large_x1_0` model of PaddleClas as an example, we will introduce how to use `paddle_lite_opt` to complete the conversion from the pre-trained model to the inference model, and then to the Paddle-Lite optimized model. + +```shell +# enter PaddleClas root directory +cd PaddleClas_root_path + +# download and uncompress the inference model +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_infer.tar +tar -xf MobileNetV3_large_x1_0_infer.tar + + +# convert inference model to Paddle-Lite optimized model +paddle_lite_opt --model_file=./MobileNetV3_large_x1_0_infer/inference.pdmodel --param_file=./MobileNetV3_large_x1_0_infer/inference.pdiparams --optimize_out=./MobileNetV3_large_x1_0 +``` + +When the above code command is completed, there will be ``MobileNetV3_large_x1_0.nb` in the current directory, which is the converted model file. + + +## 2.2 Run optimized model on Phone + +1. Prepare an Android phone with `arm8`. If the compiled inference library and `opt` file are `armv7`, you need an `arm7` phone and modify `ARM_ABI = arm7` in the Makefile. + +2. Install the ADB tool on the computer. + + * Install ADB for MAC + + Recommend use homebrew to install. + + ```shell + brew cask install android-platform-tools + ``` + * Install ADB for Linux + + ```shell + sudo apt update + sudo apt install -y wget adb + ``` + * Install ADB for windows + If install ADB fo Windows, you need to download from Google's Android platform: [Download Link](https://developer.android.com/studio). + + First, make sure the phone is connected to the computer, turn on the `USB debugging` option of the phone, and select the `file transfer` mode. Verify whether ADB is installed successfully as follows: + + ```shell + $ adb devices + + List of devices attached + 744be294 device + ``` + + If there is `device` output like the above, it means the installation was successful. + +4. Prepare optimized model, inference library files, test image and dictionary file used. + +```shell +cd PaddleClas_root_path +cd deploy/lite/ + +# prepare.sh will put the inference library files, the test image and the dictionary files in demo/cxx/clas +sh prepare.sh /{lite inference library path}/inference_lite_lib.android.armv8 + +# enter the working directory of lite demo +cd /{lite inference library path}/inference_lite_lib.android.armv8/ +cd demo/cxx/clas/ + +# copy the C++ inference dynamic library file （ie. .so) to the debug folder +cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/ +``` + +The `prepare.sh` take `PaddleClas/deploy/lite/imgs/tabby_cat.jpg` as the test image, and copy it to the `demo/cxx/clas/debug/` directory. + +You should put the model that optimized by `paddle_lite_opt` under the `demo/cxx/clas/debug/` directory. In this example, use `MobileNetV3_large_x1_0.nb` model file generated in [2.1.3](#2.1.3). + +The structure of the clas demo is as follows after the above command is completed: + +``` +demo/cxx/clas/ +|-- debug/ +| |--MobileNetV3_large_x1_0.nb class model +| |--tabby_cat.jpg test image +| |--imagenet1k_label_list.txt dictionary file +| |--libpaddle_light_api_shared.so C++ .so file +| |--config.txt config file +|-- config.txt config file +|-- image_classfication.cpp source code +|-- Makefile compile file +``` + +**NOTE**: + +* `Imagenet1k_label_list.txt` is the category mapping file of the `ImageNet1k` dataset. If use a custom category, you need to replace the category mapping file. +* `config.txt` contains the hyperparameters, as follows: + +```shell +clas_model_file ./MobileNetV3_large_x1_0.nb # path of model file +label_path ./imagenet1k_label_list.txt # path of category mapping file +resize_short_size 256 # the short side length after resize +crop_size 224 # side length used for inference after cropping + +visualize 0 # whether to visualize. If you set it to 1, an image file named 'clas_result.png' will be generated in the current directory. +``` + +5. Run Model on Phone + +```shell +# run compile to get the executable file 'clas_system' +make -j + +# move the compiled executable file to the debug folder +mv clas_system ./debug/ + +# push the debug folder to Phone +adb push debug /data/local/tmp/ + +adb shell +cd /data/local/tmp/debug +export LD_LIBRARY_PATH=/data/local/tmp/debug:$LD_LIBRARY_PATH + +# the usage of clas_system is as follows: +# ./clas_system "path of config file" "path of test image" +./clas_system ./config.txt ./tabby_cat.jpg +``` + +**NOTE**: If you make changes to the code, you need to recompile and repush the `debug ` folder to the phone. + +The result is as follows: + +![](../../images/inference_deployment/lite_demo_result.png) + + +## 3. FAQ + +Q1：If I want to change the model, do I need to go through the all process again? +A1：If you have completed the above steps, you only need to replace the `.nb` model file after replacing the model. At the same time, you may need to modify the path of `.nb` file in the config file and change the category mapping file to be compatible the model . + +Q2：How to change the test picture? +A2：Replace the test image under debug folder with the image you want to test，and then repush to the Phone again. diff --git a/src/PaddleClas/docs/en/inference_deployment/paddle_serving_deploy_en.md b/src/PaddleClas/docs/en/inference_deployment/paddle_serving_deploy_en.md new file mode 100644 index 0000000..7a60292 --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/paddle_serving_deploy_en.md @@ -0,0 +1,280 @@ +# Model Service Deployment + +## Catalogue + +- [1. Introduction](#1) +- [2. Installation of Serving](#2) +- [3. Service Deployment for Image Classification](#3) + - [3.1 Model Transformation](#3.1) + - [3.2 Service Deployment and Request](#3.2) +- [4. Service Deployment for Image Recognition](#4) + - [4.1 Model Transformation](#4.1) + - [4.2 Service Deployment and Request](#4.2) +- [5. FAQ](#5) + + +## 1. Introduction + +[Paddle Serving](https://github.com/PaddlePaddle/Serving) is designed to provide easy deployment of on-line prediction services for deep learning developers, it supports one-click deployment of industrial-grade services, highly concurrent and efficient communication between client and server, and multiple programming languages for client development. + +This section, exemplified by HTTP deployment of prediction service, describes how to deploy model services in PaddleClas with PaddleServing. Currently, only deployment on Linux platform is supported. Windows platform is not supported. + + +## 2. Installation of Serving + +It is officially recommended to use docker for the installation and environment deployment of Serving. First, pull the docker and create a Serving-based one. + +``` +docker pull paddlepaddle/serving:0.7.0-cuda10.2-cudnn7-devel +nvidia-docker run -p 9292:9292 --name test -dit paddlepaddle/serving:0.7.0-cuda10.2-cudnn7-devel bash +nvidia-docker exec -it test bash +``` + +Once you are in docker, install the Serving-related python packages. + +``` +pip3 install paddle-serving-client==0.7.0 +pip3 install paddle-serving-server==0.7.0 # CPU +pip3 install paddle-serving-app==0.7.0 +pip3 install paddle-serving-server-gpu==0.7.0.post102 #GPU with CUDA10.2 + TensorRT6 +# For other GPU environemnt, confirm the environment before choosing which one to execute +pip3 install paddle-serving-server-gpu==0.7.0.post101 # GPU with CUDA10.1 + TensorRT6 +pip3 install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUDA11.2 + TensorRT8 +``` + +- Speed up the installation process by replacing the source with `-i https://pypi.tuna.tsinghua.edu.cn/simple`. +- For other environment configuration and installation, please refer to [Install Paddle Serving using docker](https://github.com/PaddlePaddle/Serving/blob/v0.7.0/doc/Install_EN.md) +- To deploy CPU services, please install the CPU version of serving-server with the following command. + +``` +pip install paddle-serving-server +``` + + +## 3. Service Deployment for Image Classification + + +### 3.1 Model Transformation + +When adopting PaddleServing for service deployment, the saved inference model needs to be converted to a Serving model. The following part takes the classic ResNet50_vd model as an example to introduce the deployment of image classification service. + +- Enter the working directory: + +``` +cd deploy/paddleserving +``` + +- Download the inference model of ResNet50_vd： + +``` +# Download and decompress the ResNet50_vd model +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar && tar xf ResNet50_vd_infer.tar +``` + +- Convert the downloaded inference model into a format that is readily deployable by Server with the help of paddle_serving_client. + +``` +# Convert the ResNet50_vd model +python3 -m paddle_serving_client.convert --dirname ./ResNet50_vd_infer/ \ + --model_filename inference.pdmodel \ + --params_filename inference.pdiparams \ + --serving_server ./ResNet50_vd_serving/ \ + --serving_client ./ResNet50_vd_client/ +``` + +After the transformation, `ResNet50_vd_serving` and `ResNet50_vd_client` will be added to the current folder in the following format: + +``` +|- ResNet50_vd_server/ + |- __model__ + |- __params__ + |- serving_server_conf.prototxt + |- serving_server_conf.stream.prototxt +|- ResNet50_vd_client + |- serving_client_conf.prototxt + |- serving_client_conf.stream.prototxt +``` + +Having obtained the model file, modify the alias name in `serving_server_conf.prototxt` under directory `ResNet50_vd_server` by changing `alias_name` in `fetch_var` to `prediction`. + +**Notes**: Serving supports input and output renaming to ensure its compatibility with the deployment of different models. In this case, modifying the alias_name of the configuration file is the only step needed to complete the inference and deployment of all kinds of models. The modified serving_server_conf.prototxt is shown below: + +``` +feed_var { + name: "inputs" + alias_name: "inputs" + is_lod_tensor: false + feed_type: 1 + shape: 3 + shape: 224 + shape: 224 +} +fetch_var { + name: "save_infer_model/scale_0.tmp_1" + alias_name: "prediction" + is_lod_tensor: true + fetch_type: 1 + shape: -1 +} +``` + + +### 3.2 Service Deployment and Request + +Paddleserving's directory contains the code to start the pipeline service and send prediction requests, including: + +``` +__init__.py +config.yml # Configuration file for starting the service +pipeline_http_client.py # Script for sending pipeline prediction requests by http +pipeline_rpc_client.py # Script for sending pipeline prediction requests by rpc +classification_web_service.py # Script for starting the pipeline server +``` + +- Start the service： + +``` +# Start the service and the run log is saved in log.txt +python3 classification_web_service.py &>log.txt & +``` + +Once the service is successfully started, a log will be printed in log.txt similar to the following ![img](../../../deploy/paddleserving/imgs/start_server.png) + +- Send request： + +``` +# Send service request +python3 pipeline_http_client.py +``` + +Once the service is successfully started, the prediction results will be printed in the cmd window, see the following example:![img](../../../deploy/paddleserving/imgs/results.png) + + + +## 4. Service Deployment for Image Recognition + +When using PaddleServing for service deployment, the saved inference model needs to be converted to a Serving model. The following part, exemplified by the ultra-lightweight model for image recognition in PP-ShiTu, details the deployment of image recognition service. + + + +## 4.1 Model Transformation + +- Download inference models for general detection and general recognition + +``` +cd deploy +# Download and decompress general recogntion models +wget -P models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar +cd models +tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar +# Download and decompress general detection models +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar +tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar +``` + +- Convert the inference model for recognition into a Serving model: + +``` +# Convert the recognition model +python3 -m paddle_serving_client.convert --dirname ./general_PPLCNet_x2_5_lite_v1.0_infer/ \ + --model_filename inference.pdmodel \ + --params_filename inference.pdiparams \ + --serving_server ./general_PPLCNet_x2_5_lite_v1.0_serving/ \ + --serving_client ./general_PPLCNet_x2_5_lite_v1.0_client/ +``` + +After the transformation, `general_PPLCNet_x2_5_lite_v1.0_serving/` and `general_PPLCNet_x2_5_lite_v1.0_serving/` will be added to the current folder. Modify the alias name in serving_server_conf.prototxt under the directory `general_PPLCNet_x2_5_lite_v1.0_serving/` by changing `alias_name` to `features` in `fetch_var`. The modified serving_server_conf.prototxt is similar to the following: + +``` +feed_var { + name: "x" + alias_name: "x" + is_lod_tensor: false + feed_type: 1 + shape: 3 + shape: 224 + shape: 224 +} +fetch_var { + name: "save_infer_model/scale_0.tmp_1" + alias_name: "features" + is_lod_tensor: true + fetch_type: 1 + shape: -1 +} +``` + +- Convert the inference model for detection into a Serving model: + +``` +# Convert the general detection model +python3 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \ + --model_filename inference.pdmodel \ + --params_filename inference.pdiparams \ + --serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \ + --serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/ +``` + +After the transformation, `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` and `picodet_PPLCNet_x2_5_ mainbody_lite_v1.0_client/` will be added to the current folder. + +**Note:** The alias name in the serving_server_conf.prototxt under the directory`picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` requires no modification. + +- Download and decompress the constructed search library index + +``` +cd ../ +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar && tar -xf drink_dataset_v1.0.tar +``` + + +## 4.2 Service Deployment and Request + +**Note:** Since the recognition service involves multiple models, PipeLine is adopted for better performance. This deployment method does not support the windows platform for now. + +- Enter the working directory + +``` +cd ./deploy/paddleserving/recognition +``` + +Paddleserving's directory contains the code to start the pipeline service and send prediction requests, including: + +``` +__init__.py +config.yml # Configuration file for starting the service +pipeline_http_client.py # Script for sending pipeline prediction requests by http +pipeline_rpc_client.py # Script for sending pipeline prediction requests by rpc +recognition_web_service.py # Script for starting the pipeline server +``` + +- Start the service： + +``` +# Start the service and the run log is saved in log.txt +python3 recognition_web_service.py &>log.txt & +``` + +Once the service is successfully started, a log will be printed in log.txt similar to the following ![img](../../../deploy/paddleserving/imgs/start_server_shitu.png) + +- Send request： + +``` +python3 pipeline_http_client.py +``` + +Once the service is successfully started, the prediction results will be printed in the cmd window, see the following example: ![img](../../../deploy/paddleserving/imgs/results_shitu.png) + + + +## 5.FAQ + +**Q1**： After sending a request, no result is returned or the output is prompted with a decoding error. + +**A1**： Please turn off the proxy before starting the service and sending requests, try the following command: + +``` +unset https_proxy +unset http_proxy +``` + +For more types of service deployment, such as `RPC prediction services`, you can refer to the [github official website](https://github.com/PaddlePaddle/Serving/tree/v0.7.0/examples) of Serving. diff --git a/src/PaddleClas/docs/en/inference_deployment/python_deploy_en.md b/src/PaddleClas/docs/en/inference_deployment/python_deploy_en.md new file mode 100644 index 0000000..22dd4e6 --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/python_deploy_en.md @@ -0,0 +1,142 @@ +# Infering based on Python prediction engine + +The inference model (the model saved by `paddle.jit.save`) is generally a solidified model saved after the model training is completed, and is mostly used to give prediction in deployment. + +The model saved during the training process is the checkpoints model, which saves the parameters of the model and is mostly used to resume training. + +Compared with the checkpoints model, the inference model will additionally save the structural information of the model. Therefore, it is easier to deploy because the model structure and model parameters are already solidified in the inference model file, and is suitable for integration with actual systems. + +Please refer to the document [install paddle](../installation/install_paddle_en.md) and [install paddleclas](../installation/install_paddleclas_en.md) to prepare the environment. + +--- + +## Catalogue + +- [1. Image classification inference](#1) +- [2. Mainbody detection model inference](#2) +- [3. Feature Extraction model inference](#3) +- [4. Concatenation of mainbody detection, feature extraction and vector search](#4) + + + +## 1. Image classification inference + +First, please refer to the document [export model](./export_model_en.md) to prepare the inference model files. All the command should be run under `deploy` folder of PaddleClas: + +```shell +cd deploy +``` + +For classification model inference, you can execute the following commands: + +```shell +python python/predict_cls.py -c configs/inference_cls.yaml +``` + +In the configuration file `configs/inference_cls.yaml`, the following fields are used to configure prediction parameters: +* `Global.infer_imgs`: The path of image to be predicted; +* `Global.inference_model_dir`: The directory of inference model files. There should be contain the model files (`inference.pdmodel` and `inference.pdiparams`); +* `Global.use_tensorrt`: Whether use `TensorRT`, `False` by default; +* `Global.use_gpu`: Whether use GPU, `True` by default; +* `Global.enable_mkldnn`: Whether use `MKL-DNN`, `False` by default. Valid only when `use_gpu` is `False`; +* `Global.use_fp16`: Whether use `FP16`, `False` by default; +* `PreProcess`: To config the preprocessing of image to be predicted; +* `PostProcess`: To config the postprocessing of prediction results; +* `PostProcess.Topk.class_id_map_file`: The path of file mapping label and class id. By default ImageNet1k (`./utils/imagenet1k_label_list.txt`). + +**Notice**: +* If VisionTransformer series models used, such as `DeiT_***_384`, `ViT_***_384`, please notice the size of model input. And you could need to specify the `PreProcess.resize_short=384`, `PreProcess.resize=384`. +* If you want to improve the speed of the evaluation, it is recommended to enable TensorRT when using GPU, and MKL-DNN when using CPU. + +```shell +python python/predict_cls.py -c configs/inference_cls.yaml -o Global.infer_imgs=images/ILSVRC2012_val_00010010.jpeg +``` + +If you want to use the CPU for prediction, you can switch value of `use_gpu` in config file to `False`. Or you can execute the command as follows +``` +python python/predict_cls.py -c configs/inference_cls.yaml -o Global.use_gpu=False +``` + + +## 2. Mainbody detection model inference + +The following will introduce the mainbody detection model inference. All the command should be run under `deploy` folder of PaddleClas: + +```shell +cd deploy +``` + +For mainbody detection model inference, you can execute the following commands: + +```shell +mkdir -p models +cd models +# download mainbody detection inference model +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar && tar -xf ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar +cd .. +# predict +python python/predict_det.py -c configs/inference_det.yaml +``` + +The input example image is as follows: +[](../images/recognition/product_demo/wangzai.jpg) + +The output will be: + +```text +[{'class_id': 0, 'score': 0.4762245, 'bbox': array([305.55115, 226.05322, 776.61084, 930.42395], dtype=float32), 'label_name': 'foreground'}] +``` + +And the visualise result is as follows: +[](../images/recognition/product_demo/wangzai_det_result.jpg) + +If you want to detect another image, you can change the value of `infer_imgs` in `configs/inference_det.yaml`, +or you can use `-o Global.infer_imgs` argument. For example, if you want to detect `images/anmuxi.jpg`: + +```shell +python python/predict_det.py -c configs/inference_det.yaml -o Global.infer_imgs=images/anmuxi.jpg +``` + +If you want to use the CPU for prediction, you can switch value of `use_gpu` in config file to `False`. Or you can execute the command as follows +``` +python python/predict_det.py -c configs/inference_det.yaml -o Global.use_gpu=False +``` + + +## 3. Feature Extraction model inference + +First, please refer to the document [export model](./export_model_en.md) to prepare the inference model files. All the command should be run under `deploy` folder of PaddleClas: + +```shell +cd deploy +``` + +For feature extraction model inference, you can execute the following commands: + +```shell +mkdir -p models +cd models +# download feature extraction inference model +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar && tar -xf product_ResNet50_vd_aliproduct_v1.0_infer.tar +cd .. +# predict +python python/predict_rec.py -c configs/inference_rec.yaml +``` +You can get a 512-dim feature printed in the command line. + +If you want to extract feature of another image, you can change the value of `infer_imgs` in `configs/inference_rec.yaml`, +or you can use `-o Global.infer_imgs` argument. For example, if you want to try `images/anmuxi.jpg`: + +```shell +python python/predict_rec.py -c configs/inference_rec.yaml -o Global.infer_imgs=images/anmuxi.jpg +``` + +If you want to use the CPU for prediction, you can switch value of `use_gpu` in config file to `False`. Or you can execute the command as follows + +``` +python python/predict_rec.py -c configs/inference_rec.yaml -o Global.use_gpu=False +``` + + +## 4. Concatenation of mainbody detection, feature extraction and vector search + Please refer to [Quick Start of Recognition](../quick_start/quick_start_recognition_en.md) diff --git a/src/PaddleClas/docs/en/inference_deployment/whl_deploy_en.md b/src/PaddleClas/docs/en/inference_deployment/whl_deploy_en.md new file mode 100644 index 0000000..224d41a --- /dev/null +++ b/src/PaddleClas/docs/en/inference_deployment/whl_deploy_en.md @@ -0,0 +1,256 @@ +# PaddleClas wheel package + +Paddleclas supports Python WHL package for prediction. At present, WHL package only supports image classification, but does not support subject detection, feature extraction and vector retrieval. + +--- + +## Catalogue + +- [1. Installation](#1) +- [2. Quick Start](#2) +- [3. Definition of Parameters](#3) +- [4. Usage](#4) + - [4.1 View help information](#4.1) + - [4.2 Prediction using inference model provide by PaddleClas](#4.2) + - [4.3 Prediction using local model files](#4.3) + - [4.4 Prediction by batch](#4.4) + - [4.5 Prediction of Internet image](#4.5) + - [4.6 Prediction of `NumPy.array` format image](#4.6) + - [4.7 Save the prediction result(s)](#4.7) + - [4.8 Specify the mapping between class id and label name](#4.8) + + +## 1. Installation + +* installing from pypi + +```bash +pip3 install paddleclas==2.2.1 +``` + +* build own whl package and install + +```bash +python3 setup.py bdist_wheel +pip3 install dist/* +``` + + +## 2. Quick Start +* Using the `ResNet50` model provided by PaddleClas, the following image(`'docs/images/inference_deployment/whl_demo.jpg'`) as an example. + +![](../../images/inference_deployment/whl_demo.jpg) + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs='docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +**Note**: `PaddleClas.predict()` is a `generator`. Therefore you need to use `next()` or `for` call it iteratively. It will perform a prediction by `batch_size` and return the prediction result(s) when called. Examples of returned results are as follows: + +``` +>>> result +[{'class_ids': [8, 7, 136, 80, 84], 'scores': [0.79368, 0.16329, 0.01853, 0.00959, 0.00239], 'label_names': ['hen', 'cock', 'European gallinule, Porphyrio porphyrio', 'black grouse', 'peacock']}] +``` + +* CLI +```bash +paddleclas --model_name=ResNet50 --infer_imgs="docs/images/inference_deployment/whl_demo.jpg" +``` + +``` +>>> result +filename: docs/images/inference_deployment/whl_demo.jpg, top-5, class_ids: [8, 7, 136, 80, 84], scores: [0.79368, 0.16329, 0.01853, 0.00959, 0.00239], label_names: ['hen', 'cock', 'European gallinule, Porphyrio porphyrio', 'black grouse', 'peacock'] +Predict complete! +``` + + +## 3. Definition of Parameters + +The following parameters can be specified in Command Line or used as parameters of the constructor when instantiating the PaddleClas object in Python. +* model_name(str): If using inference model based on ImageNet1k provided by Paddle, please specify the model's name by the parameter. +* inference_model_dir(str): Local model files directory, which is valid when `model_name` is not specified. The directory should contain `inference.pdmodel` and `inference.pdiparams`. +* infer_imgs(str): The path of image to be predicted, or the directory containing the image files, or the URL of the image from Internet. +* use_gpu(bool): Whether to use GPU or not, default by `True`. +* gpu_mem(int): GPU memory usages，default by `8000`。 +* use_tensorrt(bool): Whether to open TensorRT or not. Using it can greatly promote predict preformance, default by `False`. +* enable_mkldnn(bool): Whether enable MKLDNN or not, default `False`. +* cpu_num_threads(int): Assign number of cpu threads, valid when `--use_gpu` is `False` and `--enable_mkldnn` is `True`, default by `10`. +* batch_size(int): Batch size, default by `1`. +* resize_short(int): Resize the minima between height and width into `resize_short`, default by `256`. +* crop_size(int): Center crop image to `crop_size`, default by `224`. +* topk(int): Print (return) the `topk` prediction results, default by `5`. +* class_id_map_file(str): The mapping file between class ID and label, default by `ImageNet1K` dataset's mapping. +* pre_label_image(bool): whether prelabel or not, default=False. +* save_dir(str): The directory to save the prediction results that can be used as pre-label, default by `None`, that is, not to save. + +**Note**: If you want to use `Transformer series models`, such as `DeiT_***_384`, `ViT_***_384`, etc., please pay attention to the input size of model, and need to set `resize_short=384`, `resize=384`. The following is a demo. + +* CLI: +```bash +from paddleclas import PaddleClas, get_default_confg +paddleclas --model_name=ViT_base_patch16_384 --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' --resize_short=384 --crop_size=384 +``` + +* Python: +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ViT_base_patch16_384', resize_short=384, crop_size=384) +``` + + +## 4. Usage + +PaddleClas provides two ways to use: +1. Python interative programming; +2. Bash command line programming. + + +### 4.1 View help information + +* CLI +```bash +paddleclas -h +``` + + +### 4.2 Prediction using inference model provide by PaddleClas +You can use the inference model provided by PaddleClas to predict, and only need to specify `model_name`. In this case, PaddleClas will automatically download files of specified model and save them in the directory `~/.paddleclas/`. + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs = 'docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' +``` + + +### 4.3 Prediction using local model files +You can use the local model files trained by yourself to predict, and only need to specify `inference_model_dir`. Note that the directory must contain `inference.pdmodel` and `inference.pdiparams`. + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(inference_model_dir='./inference/') +infer_imgs = 'docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --inference_model_dir='./inference/' --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' +``` + + +### 4.4 Prediction by batch +You can predict by batch, only need to specify `batch_size` when `infer_imgs` is direcotry contain image files. + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50', batch_size=2) +infer_imgs = 'docs/images/' +result=clas.predict(infer_imgs) +for r in result: + print(r) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/' --batch_size 2 +``` + + +### 4.5 Prediction of Internet image +You can predict the Internet image, only need to specify URL of Internet image by `infer_imgs`. In this case, the image file will be downloaded and saved in the directory `~/.paddleclas/images/`. + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs = 'https://raw.githubusercontent.com/paddlepaddle/paddleclas/release/2.2/docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='https://raw.githubusercontent.com/paddlepaddle/paddleclas/release/2.2/docs/images/inference_deployment/whl_demo.jpg' +``` + + +### 4.6 Prediction of NumPy.array format image +In Python code, you can predict the `NumPy.array` format image, only need to use the `infer_imgs` to transfer variable of image data. Note that the models in PaddleClas only support to predict 3 channels image data, and channels order is `RGB`. + +* python +```python +import cv2 +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs = cv2.imread("docs/en/inference_deployment/whl_deploy_en.md")[:, :, ::-1] +result=clas.predict(infer_imgs) +print(next(result)) +``` + + +### 4.7 Save the prediction result(s) +You can save the prediction result(s) as pre-label, only need to use `pre_label_out_dir` to specify the directory to save. + +* python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50', save_dir='./output_pre_label/') +infer_imgs = 'docs/images/inference_deployment/whl_' # it can be infer_imgs folder path which contains all of images you want to predict. +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/inference_deployment/whl_' --save_dir='./output_pre_label/' +``` + + +### 4.8 Specify the mapping between class id and label name +You can specify the mapping between class id and label name, only need to use `class_id_map_file` to specify the mapping file. PaddleClas uses ImageNet1K's mapping by default. + +The content format of mapping file shall be: + +``` +class_idclass_name<\n> +``` + +For example: + +``` +0 tench, Tinca tinca +1 goldfish, Carassius auratus +2 great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias +...... +``` + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50', class_id_map_file='./ppcls/utils/imagenet1k_label_list.txt') +infer_imgs = 'docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' --class_id_map_file='./ppcls/utils/imagenet1k_label_list.txt' +``` diff --git a/src/PaddleClas/docs/en/installation/index.rst b/src/PaddleClas/docs/en/installation/index.rst new file mode 100644 index 0000000..39d432a --- /dev/null +++ b/src/PaddleClas/docs/en/installation/index.rst @@ -0,0 +1,8 @@ +installation +================================ + +.. toctree:: + :maxdepth: 2 + + install_paddle_en.md + install_paddleclas_en.md diff --git a/src/PaddleClas/docs/en/installation/install_paddle_en.md b/src/PaddleClas/docs/en/installation/install_paddle_en.md new file mode 100644 index 0000000..c282f3e --- /dev/null +++ b/src/PaddleClas/docs/en/installation/install_paddle_en.md @@ -0,0 +1,100 @@ +# Install PaddlePaddle + +--- + +## Catalogue + +- [1. Environment requirements](#1) +- [2.(Recommended) Prepare a docker environment](#2) +- [3. Install PaddlePaddle using pip](#3) +- [4. Verify installation](#4) + +At present, **PaddleClas** requires **PaddlePaddle** version `>=2.0`. Docker is recomended to run Paddleclas, for more detailed information about docker and nvidia-docker, you can refer to the [tutorial](https://docs.docker.com/get-started/). If you do not want to use docker, you can skip section [2. (Recommended) Prepare a docker environment](#2), and go into section [3. Install PaddlePaddle using pip](#3). + + + +## 1. Environment requirements + +- python 3.x +- cuda >= 10.1 (necessary if paddlepaddle-gpu is used) +- cudnn >= 7.6.4 (necessary if paddlepaddle-gpu is used) +- nccl >= 2.1.2 (necessary distributed training/eval is used) +- gcc >= 8.2 + +**Recomends**: +* When CUDA version is 10.1, the driver version `>= 418.39`; +* When CUDA version is 10.2, the driver version `>= 440.33`; +* For more CUDA versions and specific driver versions, please refer to [link](https://docs.nvidia.com/deploy/cuda-compatibility/index.html). + + + +## 2. (Recommended) Prepare a docker environment + +* Switch to the working directory + +```shell +cd /home/Projects +``` + +* Create docker container +The following commands will create a docker container named ppcls and map the current working directory to the `/paddle' directory in the container. + +```shell +# For GPU users +sudo nvidia-docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it paddlepaddle/paddle:2.1.0-gpu-cuda10.2-cudnn7 /bin/bash + +# For CPU users +sudo docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it paddlepaddle/paddle:2.1.0 /bin/bash +``` + +**Notices**: +* The first time you use this docker image, it will be downloaded automatically. Please be patient; +* The above command will create a docker container named ppcls, and there is no need to run the command again when using the container again; +* The parameter `--shm-size=8g` will set the shared memory of the container to 8g. If conditions permit, it is recommended to set this parameter to a larger value, such as `64g`; +* You can also access [DockerHub](https://hub.Docker.com/r/paddlepaddle/paddle/tags/) to obtain the image adapted to your machine; +* Exit / Enter the docker container: + * After entering the docker container, you can exit the current container by pressing `Ctrl + P + Q` without closing the container; + * To re-enter the container, use the following command: + ```shell + sudo Docker exec -it ppcls /bin/bash + ``` + + + +## 3. Install PaddlePaddle using pip + +If you want to use PaddlePaddle on GPU, you can use the following command to install PaddlePaddle. + +```bash +pip install paddlepaddle-gpu --upgrade -i https://mirror.baidu.com/pypi/simple +``` + +If you want to use PaddlePaddle on CPU, you can use the following command to install PaddlePaddle. + +```bash +pip install paddlepaddle --upgrade -i https://mirror.baidu.com/pypi/simple +``` + +**Note:** +* If you have already installed CPU version of PaddlePaddle and want to use GPU version now, you should uninstall CPU version of PaddlePaddle and then install GPU version to avoid package confusion. +* You can also compile PaddlePaddle from source code, please refer to [PaddlePaddle Installation tutorial](http://www.paddlepaddle.org.cn/install/quick) to more compilation options. + + + +## 4. Verify Installation + +```python +import paddle +paddle.utils.run_check() +``` + +Check PaddlePaddle version： + +```bash +python -c "import paddle; print(paddle.__version__)" +``` + +Note: +* Make sure the compiled source code is later than PaddlePaddle2.0. +* Indicate `WITH_DISTRIBUTE=ON` when compiling, Please refer to [Instruction](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#id3) for more details. +* When running in docker, in order to ensure that the container has enough shared memory for dataloader acceleration of Paddle, please set the parameter `--shm-size=8g` at creating a docker container, if conditions permit, you can set it to a larger value. diff --git a/src/PaddleClas/docs/en/installation/install_paddleclas_en.md b/src/PaddleClas/docs/en/installation/install_paddleclas_en.md new file mode 100644 index 0000000..332db69 --- /dev/null +++ b/src/PaddleClas/docs/en/installation/install_paddleclas_en.md @@ -0,0 +1,32 @@ +# Install PaddleClas + +--- + +## Catalogue + +* [1. Clone PaddleClas source code](#1) +* [2. Install requirements](#2) + + + +### 1. Clone PaddleClas source code + +```shell +git clone https://github.com/PaddlePaddle/PaddleClas.git -b develop +``` + +If it is too slow for you to download from github, you can download PaddleClas from gitee. The command is as follows. + +```shell +git clone https://gitee.com/paddlepaddle/PaddleClas.git -b develop +``` + + + +## 2. Install requirements + +PaddleClas dependencies are listed in file `requirements.txt`, you can use the following command to install the dependencies. + +``` +pip install --upgrade -r requirements.txt -i https://mirror.baidu.com/pypi/simple +``` diff --git a/src/PaddleClas/docs/en/introduction/function_intro_en.md b/src/PaddleClas/docs/en/introduction/function_intro_en.md new file mode 100644 index 0000000..1a1a54b --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/function_intro_en.md @@ -0,0 +1,13 @@ +## Features of PaddleClas + +PaddleClas is an image recognition toolset for industry and academia, helping users train better computer vision models and apply them in real scenarios. Specifically, it contains the following core features. + +- Practical image recognition system: Integrate detection, feature learning, and retrieval modules to be applicable to all types of image recognition tasks. Four sample solutions are provided, including product recognition, vehicle recognition, logo recognition, and animation character recognition. +- Rich library of pre-trained models: Provide a total of 175 ImageNet pre-trained models of 36 series, among which 7 selected series of models support fast structural modification. +- Comprehensive and easy-to-use feature learning components: 12 metric learning methods are integrated and can be combined and switched at will through configuration files. +- SSLD knowledge distillation: The 14 classification pre-training models generally improved their accuracy by more than 3%; among them, the ResNet50_vd model achieved a Top-1 accuracy of 84.0% on the Image-Net-1k dataset and the Res2Net200_vd pre-training model achieved a Top-1 accuracy of 85.1%. +- Data augmentation: Provide 8 data augmentation algorithms such as AutoAugment, Cutout, Cutmix, etc. with the detailed introduction, code replication, and evaluation of effectiveness in a unified experimental environment. + +![](../../images/recognition.gif) + +For more information about the quick start of image recognition, algorithm details, model training and evaluation, and prediction and deployment methods, please refer to the [README Tutorial](../../../README_ch.md) on home page. diff --git a/src/PaddleClas/docs/en/introduction/index.rst b/src/PaddleClas/docs/en/introduction/index.rst new file mode 100644 index 0000000..e22a647 --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/index.rst @@ -0,0 +1,8 @@ +introduction +================================ + +.. toctree:: + :maxdepth: 2 + + function_intro_en.md + more_demo/index diff --git a/src/PaddleClas/docs/en/introduction/more_demo/cartoon.md b/src/PaddleClas/docs/en/introduction/more_demo/cartoon.md new file mode 100644 index 0000000..042fbf2 --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/more_demo/cartoon.md @@ -0,0 +1,53 @@ +# Cartoon Demo Images + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/en/introduction/more_demo/index.rst b/src/PaddleClas/docs/en/introduction/more_demo/index.rst new file mode 100644 index 0000000..f09bccf --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/more_demo/index.rst @@ -0,0 +1,11 @@ +more_demo +================================ + +.. toctree:: + :maxdepth: 1 + + product.md + logo.md + cartoon.md + more_demo.md + vehicle.md diff --git a/src/PaddleClas/docs/en/introduction/more_demo/logo.md b/src/PaddleClas/docs/en/introduction/more_demo/logo.md new file mode 100644 index 0000000..0b8ee1b --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/more_demo/logo.md @@ -0,0 +1,65 @@ +# Logo Demo Images + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/en/introduction/more_demo/more_demo.md b/src/PaddleClas/docs/en/introduction/more_demo/more_demo.md new file mode 100644 index 0000000..1e8d00d --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/more_demo/more_demo.md @@ -0,0 +1,34 @@ +## Demo images +- Product recognition +
+
+
+
+
+ +[More demo images](product.md) + +- Cartoon character recognition +
+
+
+ +[More demo images](cartoon.md) + +- Logo recognition +
+
+
+
+ +
+
+ +[More demo images](logo.md) + +- Car recognition +
+
+
+ +[More demo images](vehicle.md) diff --git a/src/PaddleClas/docs/en/introduction/more_demo/product.md b/src/PaddleClas/docs/en/introduction/more_demo/product.md new file mode 100644 index 0000000..f80203f --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/more_demo/product.md @@ -0,0 +1,179 @@ +# Product Demo Images + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/en/introduction/more_demo/vehicle.md b/src/PaddleClas/docs/en/introduction/more_demo/vehicle.md new file mode 100644 index 0000000..2f37e39 --- /dev/null +++ b/src/PaddleClas/docs/en/introduction/more_demo/vehicle.md @@ -0,0 +1,33 @@ +# Vehicle Demo Images + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/en/make.bat b/src/PaddleClas/docs/en/make.bat new file mode 100644 index 0000000..922152e --- /dev/null +++ b/src/PaddleClas/docs/en/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=. +set BUILDDIR=_build + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/src/PaddleClas/docs/en/models/DLA_en.md b/src/PaddleClas/docs/en/models/DLA_en.md new file mode 100644 index 0000000..0d7b838 --- /dev/null +++ b/src/PaddleClas/docs/en/models/DLA_en.md @@ -0,0 +1,27 @@ +# DLA series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## Overview + +DLA (Deep Layer Aggregation). Visual recognition requires rich representations that span levels from low to high, scales from small to large, and resolutions from fine to coarse. Even with the depth of features in a convolutional network, a layer in isolation is not enough: compounding and aggregating these representations improves inference of what and where. Although skip connections have been incorporated to combine layers, these connections have been "shallow" themselves, and only fuse by simple, one-step operations. The authors augment standard architectures with deeper aggregation to better fuse information across layers. Deep layer aggregation structures iteratively and hierarchically merge the feature hierarchy to make networks with better accuracy and fewer parameters. Experiments across architectures and tasks show that deep layer aggregation improves recognition and resolution compared to existing branching and merging schemes. [paper](https://arxiv.org/abs/1707.06484) + + +## 2. Accuracy, FLOPs and Parameters + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:-----------------:|:----------:|:---------:|:---------:|:---------:| +| DLA34 | 15.8 | 3.1 | 76.03 | 92.98 | +| DLA46_c | 1.3 | 0.5 | 63.21 | 85.30 | +| DLA46x_c | 1.1 | 0.5 | 64.36 | 86.01 | +| DLA60 | 22.0 | 4.2 | 76.10 | 92.92 | +| DLA60x | 17.4 | 3.5 | 77.53 | 93.78 | +| DLA60x_c | 1.3 | 0.6 | 66.45 | 87.54 | +| DLA102 | 33.3 | 7.2 | 78.93 | 94.52 | +| DLA102x | 26.4 | 5.9 | 78.10 | 94.00 | +| DLA102x2 | 41.4 | 9.3 | 78.85 | 94.45 | +| DLA169 | 53.5 | 11.6 | 78.09 | 94.09 | diff --git a/src/PaddleClas/docs/en/models/DPN_DenseNet_en.md b/src/PaddleClas/docs/en/models/DPN_DenseNet_en.md new file mode 100644 index 0000000..8976766 --- /dev/null +++ b/src/PaddleClas/docs/en/models/DPN_DenseNet_en.md @@ -0,0 +1,78 @@ +# DPN and DenseNet series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed based on V100 GPU](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +DenseNet is a new network structure proposed in 2017 and was the best paper of CVPR. The network has designed a new cross-layer connected block called dense-block. Compared to the bottleneck in ResNet, dense-block has designed a more aggressive dense connection module, that is, connecting all the layers to each other, and each layer will accept all the layers in front of it as its additional input. DenseNet stacks all dense-blocks into a densely connected network. The dense connection makes DenseNet easier to backpropagate, making the network easier to train and converge. The full name of DPN is Dual Path Networks, which is a network composed of DenseNet and ResNeXt, which proves that DenseNet can extract new features from the previous level, and ResNeXt essentially reuses the extracted features . The author further analyzes and finds that ResNeXt has high reuse rate for features, but low redundancy, while DenseNet can create new features, but with high redundancy. Combining the advantages of the two structures, the author designed the DPN network. In the end, the DPN network achieved better results than ResNeXt and DenseNet under the same FLOPs and parameters. + +The FLOPs, parameters, and inference time on the T4 GPU of this series of models are shown in the figure below. + +![](../../images/models/T4_benchmark/t4.fp32.bs4.DPN.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.DPN.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.DPN.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.DPN.png) + +The pretrained models of these two types of models (a total of 10) are open sourced in PaddleClas at present. The indicators are shown in the figure above. It is easy to observe that under the same FLOPs and parameters, DPN has higher accuracy than DenseNet. However,because DPN has more branches, its inference speed is slower than DenseNet. Since DenseNet264 has the deepest layers in all DenseNet networks, it has the largest parameters,DenseNet161 has the largest width, resulting the largest FLOPs and the highest accuracy in this series. From the perspective of inference speed, DenseNet161, which has a large FLOPs and high accuracy, has a faster speed than DenseNet264, so it has a greater advantage than DenseNet264. + +For DPN series networks, the larger the model's FLOPs and parameters, the higher the model's accuracy. Among them, since the width of DPN107 is the largest, it has the largest number of parameters and FLOPs in this series of networks. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| DenseNet121 | 0.757 | 0.926 | 0.750 | | 5.690 | 7.980 | +| DenseNet161 | 0.786 | 0.941 | 0.778 | | 15.490 | 28.680 | +| DenseNet169 | 0.768 | 0.933 | 0.764 | | 6.740 | 14.150 | +| DenseNet201 | 0.776 | 0.937 | 0.775 | | 8.610 | 20.010 | +| DenseNet264 | 0.780 | 0.939 | 0.779 | | 11.540 | 33.370 | +| DPN68 | 0.768 | 0.934 | 0.764 | 0.931 | 4.030 | 10.780 | +| DPN92 | 0.799 | 0.948 | 0.793 | 0.946 | 12.540 | 36.290 | +| DPN98 | 0.806 | 0.951 | 0.799 | 0.949 | 22.220 | 58.460 | +| DPN107 | 0.809 | 0.953 | 0.802 | 0.951 | 35.060 | 82.970 | +| DPN131 | 0.807 | 0.951 | 0.801 | 0.949 | 30.510 | 75.360 | + + + + +## 3. Inference speed based on V100 GPU + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | +|-------------|-----------|-------------------|--------------------------| +| DenseNet121 | 224 | 256 | 4.371 | +| DenseNet161 | 224 | 256 | 8.863 | +| DenseNet169 | 224 | 256 | 6.391 | +| DenseNet201 | 224 | 256 | 8.173 | +| DenseNet264 | 224 | 256 | 11.942 | +| DPN68 | 224 | 256 | 11.805 | +| DPN92 | 224 | 256 | 17.840 | +| DPN98 | 224 | 256 | 21.057 | +| DPN107 | 224 | 256 | 28.685 | +| DPN131 | 224 | 256 | 28.083 | + + + +## 4. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| DenseNet121 | 224 | 256 | 4.16436 | 7.2126 | 10.50221 | 4.40447 | 9.32623 | 15.25175 | +| DenseNet161 | 224 | 256 | 9.27249 | 14.25326 | 20.19849 | 10.39152 | 22.15555 | 35.78443 | +| DenseNet169 | 224 | 256 | 6.11395 | 10.28747 | 13.68717 | 6.43598 | 12.98832 | 20.41964 | +| DenseNet201 | 224 | 256 | 7.9617 | 13.4171 | 17.41949 | 8.20652 | 17.45838 | 27.06309 | +| DenseNet264 | 224 | 256 | 11.70074 | 19.69375 | 24.79545 | 12.14722 | 26.27707 | 40.01905 | +| DPN68 | 224 | 256 | 11.7827 | 13.12652 | 16.19213 | 11.64915 | 12.82807 | 18.57113 | +| DPN92 | 224 | 256 | 18.56026 | 20.35983 | 29.89544 | 18.15746 | 23.87545 | 38.68821 | +| DPN98 | 224 | 256 | 21.70508 | 24.7755 | 40.93595 | 21.18196 | 33.23925 | 62.77751 | +| DPN107 | 224 | 256 | 27.84462 | 34.83217 | 60.67903 | 27.62046 | 52.65353 | 100.11721 | +| DPN131 | 224 | 256 | 28.58941 | 33.01078 | 55.65146 | 28.33119 | 46.19439 | 89.24904 | diff --git a/src/PaddleClas/docs/en/models/ESNet_en.md b/src/PaddleClas/docs/en/models/ESNet_en.md new file mode 100644 index 0000000..fbfcb5d --- /dev/null +++ b/src/PaddleClas/docs/en/models/ESNet_en.md @@ -0,0 +1,23 @@ +# ESNet Series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +ESNet (Enhanced ShuffleNet) is a lightweight network developed by Baidu. This network combines the advantages of MobileNetV3, GhostNet, and PPLCNet on the basis of ShuffleNetV2 to form a faster and more accurate network on ARM devices, Because of its excellent performance, [PP-PicoDet](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/picodet) launched in PaddleDetection uses this model as a backbone, with stronger object detection algorithm, the final mAP index refreshed the SOTA index of the object detection model on the ARM device in one fell swoop. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | FLOPs
(M) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:| +| ESNet_x0_25 | 62.48 | 83.46 | 30.9 | 2.83 | +| ESNet_x0_5 | 68.82 | 88.04 | 67.3 | 3.25 | +| ESNet_x0_75 | 72.24 | 90.45 | 123.7 | 3.87 | +| ESNet_x1_0 | 73.92 | 91.40 | 197.3 | 4.64 | + +Please stay tuned for information such as Inference speed. diff --git a/src/PaddleClas/docs/en/models/EfficientNet_and_ResNeXt101_wsl_en.md b/src/PaddleClas/docs/en/models/EfficientNet_and_ResNeXt101_wsl_en.md new file mode 100644 index 0000000..cd9a2d2 --- /dev/null +++ b/src/PaddleClas/docs/en/models/EfficientNet_and_ResNeXt101_wsl_en.md @@ -0,0 +1,91 @@ +# EfficientNet and ResNeXt101_wsl series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed based on V100 GPU](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +EfficientNet is a lightweight NAS-based network released by Google in 2019. EfficientNetB7 refreshed the classification accuracy of ImageNet-1k at that time. In this paper, the author points out that the traditional methods to improve the performance of neural networks mainly start with the width of the network, the depth of the network, and the resolution of the input picture. +However, the author found that balancing these three dimensions is essential for improving accuracy and efficiency through experiments. +Therefore, the author summarized how to balance the three dimensions at the same time through a series of experiments. +At the same time, based on this scaling method, the author built a total of 7 networks B1-B7 in the EfficientNet series on the basis of EfficientNetB0, and with the same FLOPs and parameters, the accuracy reached state-of-the-art effect. + +ResNeXt is an improved version of ResNet that proposed by Facebook in 2016. In 2019, Facebook researchers studied the accuracy limit of the series network on ImageNet through weakly-supervised-learning. In order to distinguish the previous ResNeXt network, the suffix of this series network is WSL, where WSL is the abbreviation of weakly-supervised-learning. In order to have stronger feature extraction capability, the researchers further enlarged the network width, among which the largest ResNeXt101_32x48d_wsl has 800 million parameters. It was trained under 940 million weak-labeled images, and the results were finetune trained on imagenet-1k. Finally, the acc-1 of imagenet-1k reaches 85.4%, which is also the network with the highest precision under the resolution of 224x224 on imagenet-1k so far. In Fix-ResNeXt, the author used a larger image resolution, made a special Fix strategy for the inconsistency of image data preprocessing in training and testing, and made ResNeXt101_32x48d_wsl have a higher accuracy. Since it used the Fix strategy, it was named Fix-ResNeXt101_32x48d_wsl. + +The FLOPs, parameters, and inference time on the T4 GPU of this series of models are shown in the figure below. + +![](../../images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs1.EfficientNet.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs1.EfficientNet.png) + +At present, there are a total of 14 pretrained models of the two types of models that PaddleClas open source. It can be seen from the above figure that the advantages of the EfficientNet series network are very obvious. The ResNeXt101_wsl series model uses more data, and the final accuracy is also higher. EfficientNet_B0_small removes SE_block based on EfficientNet_B0, which has faster inference speed. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ResNeXt101_
32x8d_wsl | 0.826 | 0.967 | 0.822 | 0.964 | 29.140 | 78.440 | +| ResNeXt101_
32x16d_wsl | 0.842 | 0.973 | 0.842 | 0.972 | 57.550 | 152.660 | +| ResNeXt101_
32x32d_wsl | 0.850 | 0.976 | 0.851 | 0.975 | 115.170 | 303.110 | +| ResNeXt101_
32x48d_wsl | 0.854 | 0.977 | 0.854 | 0.976 | 173.580 | 456.200 | +| Fix_ResNeXt101_
32x48d_wsl | 0.863 | 0.980 | 0.864 | 0.980 | 354.230 | 456.200 | +| EfficientNetB0 | 0.774 | 0.933 | 0.773 | 0.935 | 0.720 | 5.100 | +| EfficientNetB1 | 0.792 | 0.944 | 0.792 | 0.945 | 1.270 | 7.520 | +| EfficientNetB2 | 0.799 | 0.947 | 0.803 | 0.950 | 1.850 | 8.810 | +| EfficientNetB3 | 0.812 | 0.954 | 0.817 | 0.956 | 3.430 | 11.840 | +| EfficientNetB4 | 0.829 | 0.962 | 0.830 | 0.963 | 8.290 | 18.760 | +| EfficientNetB5 | 0.836 | 0.967 | 0.837 | 0.967 | 19.510 | 29.610 | +| EfficientNetB6 | 0.840 | 0.969 | 0.842 | 0.968 | 36.270 | 42.000 | +| EfficientNetB7 | 0.843 | 0.969 | 0.844 | 0.971 | 72.350 | 64.920 | +| EfficientNetB0_
small | 0.758 | 0.926 | | | 0.720 | 4.650 | + + +## 3. Inference speed based on V100 GPU + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | +|-------------------------------|-----------|-------------------|--------------------------| +| ResNeXt101_
32x8d_wsl | 224 | 256 | 19.127 | +| ResNeXt101_
32x16d_wsl | 224 | 256 | 23.629 | +| ResNeXt101_
32x32d_wsl | 224 | 256 | 40.214 | +| ResNeXt101_
32x48d_wsl | 224 | 256 | 59.714 | +| Fix_ResNeXt101_
32x48d_wsl | 320 | 320 | 82.431 | +| EfficientNetB0 | 224 | 256 | 2.449 | +| EfficientNetB1 | 240 | 272 | 3.547 | +| EfficientNetB2 | 260 | 292 | 3.908 | +| EfficientNetB3 | 300 | 332 | 5.145 | +| EfficientNetB4 | 380 | 412 | 7.609 | +| EfficientNetB5 | 456 | 488 | 12.078 | +| EfficientNetB6 | 528 | 560 | 18.381 | +| EfficientNetB7 | 600 | 632 | 27.817 | +| EfficientNetB0_
small | 224 | 256 | 1.692 | + + + +## 4. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|---------------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| ResNeXt101_
32x8d_wsl | 224 | 256 | 18.19374 | 21.93529 | 34.67802 | 18.52528 | 34.25319 | 67.2283 | +| ResNeXt101_
32x16d_wsl | 224 | 256 | 18.52609 | 36.8288 | 62.79947 | 25.60395 | 71.88384 | 137.62327 | +| ResNeXt101_
32x32d_wsl | 224 | 256 | 33.51391 | 70.09682 | 125.81884 | 54.87396 | 160.04337 | 316.17718 | +| ResNeXt101_
32x48d_wsl | 224 | 256 | 50.97681 | 137.60926 | 190.82628 | 99.01698256 | 315.91261 | 551.83695 | +| Fix_ResNeXt101_
32x48d_wsl | 320 | 320 | 78.62869 | 191.76039 | 317.15436 | 160.0838242 | 595.99296 | 1151.47384 | +| EfficientNetB0 | 224 | 256 | 3.40122 | 5.95851 | 9.10801 | 3.442 | 6.11476 | 9.3304 | +| EfficientNetB1 | 240 | 272 | 5.25172 | 9.10233 | 14.11319 | 5.3322 | 9.41795 | 14.60388 | +| EfficientNetB2 | 260 | 292 | 5.91052 | 10.5898 | 17.38106 | 6.29351 | 10.95702 | 17.75308 | +| EfficientNetB3 | 300 | 332 | 7.69582 | 16.02548 | 27.4447 | 7.67749 | 16.53288 | 28.5939 | +| EfficientNetB4 | 380 | 412 | 11.55585 | 29.44261 | 53.97363 | 12.15894 | 30.94567 | 57.38511 | +| EfficientNetB5 | 456 | 488 | 19.63083 | 56.52299 | - | 20.48571 | 61.60252 | - | +| EfficientNetB6 | 528 | 560 | 30.05911 | - | - | 32.62402 | - | - | +| EfficientNetB7 | 600 | 632 | 47.86087 | - | - | 53.93823 | - | - | +| EfficientNetB0_small | 224 | 256 | 2.39166 | 4.36748 | 6.96002 | 2.3076 | 4.71886 | 7.21888 | diff --git a/src/PaddleClas/docs/en/models/HRNet_en.md b/src/PaddleClas/docs/en/models/HRNet_en.md new file mode 100644 index 0000000..f6cf537 --- /dev/null +++ b/src/PaddleClas/docs/en/models/HRNet_en.md @@ -0,0 +1,75 @@ +# HRNet series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed based on V100 GPU](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +HRNet is a brand new neural network proposed by Microsoft research Asia in 2019. Different from the previous convolutional neural network, this network can still maintain high resolution in the deep layer of the network, so the heat map of the key points predicted is more accurate, and it is also more accurate in space. In addition, the network performs particularly well in other visual tasks sensitive to resolution, such as detection and segmentation. + +The FLOPs, parameters, and inference time on the T4 GPU of this series of models are shown in the figure below. + +![](../../images/models/T4_benchmark/t4.fp32.bs4.HRNet.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.HRNet.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.HRNet.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.HRNet.png) + +At present, there are 7 pretrained models of such models open-sourced by PaddleClas, and their indicators are shown in the figure. Among them, the reason why the accuracy of the HRNet_W48_C indicator is abnormal may be due to fluctuations in training. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| HRNet_W18_C | 0.769 | 0.934 | 0.768 | 0.934 | 4.140 | 21.290 | +| HRNet_W18_C_ssld | 0.816 | 0.958 | 0.768 | 0.934 | 4.140 | 21.290 | +| HRNet_W30_C | 0.780 | 0.940 | 0.782 | 0.942 | 16.230 | 37.710 | +| HRNet_W32_C | 0.783 | 0.942 | 0.785 | 0.942 | 17.860 | 41.230 | +| HRNet_W40_C | 0.788 | 0.945 | 0.789 | 0.945 | 25.410 | 57.550 | +| HRNet_W44_C | 0.790 | 0.945 | 0.789 | 0.944 | 29.790 | 67.060 | +| HRNet_W48_C | 0.790 | 0.944 | 0.793 | 0.945 | 34.580 | 77.470 | +| HRNet_W48_C_ssld | 0.836 | 0.968 | 0.793 | 0.945 | 34.580 | 77.470 | +| HRNet_W64_C | 0.793 | 0.946 | 0.795 | 0.946 | 57.830 | 128.060 | +| SE_HRNet_W64_C_ssld | 0.847 | 0.973 | | | 57.830 | 128.970 | + + +## 3. Inference speed based on V100 GPU + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | +|-------------|-----------|-------------------|--------------------------| +| HRNet_W18_C | 224 | 256 | 7.368 | +| HRNet_W18_C_ssld | 224 | 256 | 7.368 | +| HRNet_W30_C | 224 | 256 | 9.402 | +| HRNet_W32_C | 224 | 256 | 9.467 | +| HRNet_W40_C | 224 | 256 | 10.739 | +| HRNet_W44_C | 224 | 256 | 11.497 | +| HRNet_W48_C | 224 | 256 | 12.165 | +| HRNet_W48_C_ssld | 224 | 256 | 12.165 | +| HRNet_W64_C | 224 | 256 | 15.003 | + + + + +## 4. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| HRNet_W18_C | 224 | 256 | 6.79093 | 11.50986 | 17.67244 | 7.40636 | 13.29752 | 23.33445 | +| HRNet_W18_C_ssld | 224 | 256 | 6.79093 | 11.50986 | 17.67244 | 7.40636 | 13.29752 | 23.33445 | +| HRNet_W30_C | 224 | 256 | 8.98077 | 14.08082 | 21.23527 | 9.57594 | 17.35485 | 32.6933 | +| HRNet_W32_C | 224 | 256 | 8.82415 | 14.21462 | 21.19804 | 9.49807 | 17.72921 | 32.96305 | +| HRNet_W40_C | 224 | 256 | 11.4229 | 19.1595 | 30.47984 | 12.12202 | 25.68184 | 48.90623 | +| HRNet_W44_C | 224 | 256 | 12.25778 | 22.75456 | 32.61275 | 13.19858 | 32.25202 | 59.09871 | +| HRNet_W48_C | 224 | 256 | 12.65015 | 23.12886 | 33.37859 | 13.70761 | 34.43572 | 63.01219 | +| HRNet_W48_C_ssld | 224 | 256 | 12.65015 | 23.12886 | 33.37859 | 13.70761 | 34.43572 | 63.01219 | +| HRNet_W64_C | 224 | 256 | 15.10428 | 27.68901 | 40.4198 | 17.57527 | 47.9533 | 97.11228 | +| SE_HRNet_W64_C_ssld | 224 | 256 | 32.33651 | 69.31189 | 116.07245 | 31.69770 | 94.99546 | 174.45766 | + diff --git a/src/PaddleClas/docs/en/models/HarDNet_en.md b/src/PaddleClas/docs/en/models/HarDNet_en.md new file mode 100644 index 0000000..7388896 --- /dev/null +++ b/src/PaddleClas/docs/en/models/HarDNet_en.md @@ -0,0 +1,21 @@ +# HarDNet series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +HarDNet（Harmonic DenseNet）is a brand new neural network proposed by National Tsing Hua University in 2019, which to achieve high efficiency in terms of both low MACs and memory traffic. The new network achieves 35%, 36%, 30%, 32%, and 45% inference time reduction compared with FC-DenseNet-103, DenseNet-264, ResNet-50, ResNet-152, and SSD-VGG, respectively. We use tools including Nvidia profiler and ARM Scale-Sim to measure the memory traffic and verify that the inference latency is indeed proportional to the memory traffic consumption and the proposed network consumes low memory traffic. [Paper](https://arxiv.org/abs/1909.00948). + + +## 2. Accuracy, FLOPs and Parameters + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:---------------------:|:----------:|:---------:|:---------:|:---------:| +| HarDNet68 | 17.6 | 4.3 | 75.46 | 92.65 | +| HarDNet85 | 36.7 | 9.1 | 77.44 | 93.55 | +| HarDNet39_ds | 3.5 | 0.4 | 71.33 | 89.98 | +| HarDNet68_ds | 4.2 | 0.8 | 73.62 | 91.52 | \ No newline at end of file diff --git a/src/PaddleClas/docs/en/models/Inception_en.md b/src/PaddleClas/docs/en/models/Inception_en.md new file mode 100644 index 0000000..cf521d9 --- /dev/null +++ b/src/PaddleClas/docs/en/models/Inception_en.md @@ -0,0 +1,74 @@ +# Inception series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed based on V100 GPU](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +GoogLeNet is a new neural network structure designed by Google in 2014, which, together with VGG network, became the twin champions of the ImageNet challenge that year. GoogLeNet introduces the Inception structure for the first time, and stacks the Inception structure in the network so that the number of network layers reaches 22, which is also the mark of the convolutional network exceeding 20 layers for the first time. Since 1x1 convolution is used in the Inception structure to reduce the dimension of channel number, and Global pooling is used to replace the traditional method of processing features in multiple fc layers, the final GoogLeNet network has much less FLOPs and parameters than VGG network, which has become a beautiful scenery of neural network design at that time. + +InceptionV3 is an improvement of InceptionV2 by Google. First of all, the author optimized the Inception module in InceptionV3. At the same time, more types of Inception modules were designed and used. Further, the larger square two-dimensional convolution kernel in some Inception modules in InceptionV3 was disassembled into two smaller asymmetric convolution kernels, which can greatly save the amount of parameters. + +Xception is another improvement to InceptionV3 that Google proposed after Inception. In Xception, the author used the depthwise separable convolution to replace the traditional convolution operation, which greatly saved the network FLOPs and the number of parameters, but improved the accuracy. In DeeplabV3+, the author further improved the Xception and increased the number of Xception layers, and designed the network of Xception65 and Xception71. + +InceptionV4 is a new neural network designed by Google in 2016, when residual structure were all the rage, but the authors believe that high performance can be achieved using only Inception structure. InceptionV4 uses more Inception structure to achieve even greater precision on Imagenet-1k. + +The FLOPs, parameters, and inference time on the T4 GPU of this series of models are shown in the figure below. + +![](../../images/models/T4_benchmark/t4.fp32.bs4.Inception.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.Inception.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.Inception.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.Inception.png) + +The figure above reflects the relationship between the accuracy of Xception series and InceptionV4 and other indicators. Among them, Xception_deeplab is consistent with the structure of the paper, and Xception is an improved model developed by PaddleClas, which improves the accuracy by about 0.6% when the inference speed is basically unchanged. Details of the improved model are being updated, so stay tuned. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| GoogLeNet | 0.707 | 0.897 | 0.698 | | 2.880 | 8.460 | +| Xception41 | 0.793 | 0.945 | 0.790 | 0.945 | 16.740 | 22.690 | +| Xception41
_deeplab | 0.796 | 0.944 | | | 18.160 | 26.730 | +| Xception65 | 0.810 | 0.955 | | | 25.950 | 35.480 | +| Xception65
_deeplab | 0.803 | 0.945 | | | 27.370 | 39.520 | +| Xception71 | 0.811 | 0.955 | | | 31.770 | 37.280 | +| InceptionV3 | 0.791 | 0.946 | 0.788 | 0.944 | 11.460 | 23.830 | +| InceptionV4 | 0.808 | 0.953 | 0.800 | 0.950 | 24.570 | 42.680 | + + + +## 3. Inference speed based on V100 GPU + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | +|------------------------|-----------|-------------------|--------------------------| +| GoogLeNet | 224 | 256 | 1.807 | +| Xception41 | 299 | 320 | 3.972 | +| Xception41_
deeplab | 299 | 320 | 4.408 | +| Xception65 | 299 | 320 | 6.174 | +| Xception65_
deeplab | 299 | 320 | 6.464 | +| Xception71 | 299 | 320 | 6.782 | +| InceptionV4 | 299 | 320 | 11.141 | + + + +## 4. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|--------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| GoogLeNet | 299 | 320 | 1.75451 | 3.39931 | 4.71909 | 1.88038 | 4.48882 | 6.94035 | +| Xception41 | 299 | 320 | 2.91192 | 7.86878 | 15.53685 | 4.96939 | 17.01361 | 32.67831 | +| Xception41_
deeplab | 299 | 320 | 2.85934 | 7.2075 | 14.01406 | 5.33541 | 17.55938 | 33.76232 | +| Xception65 | 299 | 320 | 4.30126 | 11.58371 | 23.22213 | 7.26158 | 25.88778 | 53.45426 | +| Xception65_
deeplab | 299 | 320 | 4.06803 | 9.72694 | 19.477 | 7.60208 | 26.03699 | 54.74724 | +| Xception71 | 299 | 320 | 4.80889 | 13.5624 | 27.18822 | 8.72457 | 31.55549 | 69.31018 | +| InceptionV3 | 299 | 320 | 3.67502 | 6.36071 | 9.82645 | 6.64054 | 13.53630 | 22.17355 | +| InceptionV4 | 299 | 320 | 9.50821 | 13.72104 | 20.27447 | 12.99342 | 25.23416 | 43.56121 | diff --git a/src/PaddleClas/docs/en/models/LeViT_en.md b/src/PaddleClas/docs/en/models/LeViT_en.md new file mode 100644 index 0000000..6cc7245 --- /dev/null +++ b/src/PaddleClas/docs/en/models/LeViT_en.md @@ -0,0 +1,24 @@ +# LeViT series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview +LeViT is a fast inference hybrid neural network for image classification tasks. Its design considers the performance of the network model on different hardware platforms, so it can better reflect the real scenarios of common applications. Through a large number of experiments, the author found a better way to combine the convolutional neural network and the Transformer system, and proposed an attention-based method to integrate the position information encoding in the Transformer. [Paper](https://arxiv.org/abs/2104.01136)。 + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(M) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| LeViT-128S | 0.7598 | 0.9269 | 0.766 | 0.929 | 305 | 7.8 | +| LeViT-128 | 0.7810 | 0.9371 | 0.786 | 0.940 | 406 | 9.2 | +| LeViT-192 | 0.7934 | 0.9446 | 0.800 | 0.947 | 658 | 11 | +| LeViT-256 | 0.8085 | 0.9497 | 0.816 | 0.954 | 1120 | 19 | +| LeViT-384 | 0.8191 | 0.9551 | 0.826 | 0.960 | 2353 | 39 | + + +**Note**：The difference in accuracy from Reference is due to the difference in data preprocessing and the absence of distilled head as output. diff --git a/src/PaddleClas/docs/en/models/MixNet_en.md b/src/PaddleClas/docs/en/models/MixNet_en.md new file mode 100644 index 0000000..c104731 --- /dev/null +++ b/src/PaddleClas/docs/en/models/MixNet_en.md @@ -0,0 +1,27 @@ +# MixNet series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +MixNet is a lightweight network proposed by Google. The main idea of MixNet is to explore the combination of different size of kernels. The author found that the current network has the following two problems: + +- Small convolution kernel has small receptive field and few parameters, but the accuracy is not high. +- The larger convolution kernel has larger receptive field and higher accuracy, but the parameters also increase a lot . + + In order to solve the above two problems, MDConv(mixed depthwise convolution) is proposed. In this method, different size of kernels are mixed in a convolution operation block. And based on AutoML, a series of networks called MixNets are proposed, which have achieved good results on Imagenet. [paper](https://arxiv.org/pdf/1907.09595.pdf) + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | FLOPs
(M) | Params
(G | +| :------: | :---: | :---: | :---------------: | :----------: | ------------- | +| MixNet_S | 76.28 | 92.99 | 75.8 | 252.977 | 4.167 | +| MixNet_M | 77.67 | 93.64 | 77.0 | 357.119 | 5.065 | +| MixNet_L | 78.60 | 94.37 | 78.9 | 579.017 | 7.384 | + +Inference speed and other information are coming soon. diff --git a/src/PaddleClas/docs/en/models/Mobile_en.md b/src/PaddleClas/docs/en/models/Mobile_en.md new file mode 100644 index 0000000..7834ef0 --- /dev/null +++ b/src/PaddleClas/docs/en/models/Mobile_en.md @@ -0,0 +1,158 @@ +# Mobile and Embedded Vision Applications Network series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed and storage size based on SD855](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +MobileNetV1 is a network launched by Google in 2017 for use on mobile devices or embedded devices. The network replaces the depthwise separable convolution with the traditional convolution operation, that is, the combination of depthwise convolution and pointwise convolution. Compared with the traditional convolution operation, this combination can greatly save the number of parameters and computation. At the same time, MobileNetV1 can also be used for object detection, image segmentation and other visual tasks. + +MobileNetV2 is a lightweight network proposed by Google following MobileNetV1. Compared with MobileNetV1, MobileNetV2 proposed Linear bottlenecks and Inverted residual block as a basic network structures, to constitute MobileNetV2 network architecture through stacking these basic module a lot. In the end, higher classification accuracy was achieved when FLOPs was only half of MobileNetV1. + +The ShuffleNet series network is the lightweight network structure proposed by MEGVII. So far, there are two typical structures in this series network, namely, ShuffleNetV1 and ShuffleNetV2. A Channel Shuffle operation in ShuffleNet can exchange information between groups and perform end-to-end training. In the paper of ShuffleNetV2, the author proposes four criteria for designing lightweight networks, and designs the ShuffleNetV2 network according to the four criteria and the shortcomings of ShuffleNetV1. + +MobileNetV3 is a new and lightweight network based on NAS proposed by Google in 2019. In order to further improve the effect, the activation functions of relu and sigmoid were replaced with hard_swish and hard_sigmoid activation functions, and some improved strategies were introduced to reduce the amount of network computing. + +GhosttNet is a brand-new lightweight network structure proposed by Huawei in 2020. By introducing the ghost module, the problem of redundant calculation of features in traditional deep networks is greatly alleviated, which greatly reduces the amount of network parameters and calculations. + +![](../../images/models/mobile_arm_top1.png) + +![](../../images/models/mobile_arm_storage.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.params.png) + +Currently there are 32 pretrained models of the mobile series open source by PaddleClas, and their indicators are shown in the figure below. As you can see from the picture, newer lightweight models tend to perform better, and MobileNetV3 represents the latest lightweight neural network architecture. In MobileNetV3, the author used 1x1 convolution after global-avg-pooling in order to obtain higher accuracy,this operation significantly increases the number of parameters but has little impact on the amount of computation, so if the model is evaluated from a storage perspective of excellence, MobileNetV3 does not have much advantage, but because of its smaller computation, it has a faster inference speed. In addition, the SSLD distillation model in our model library performs excellently, refreshing the accuracy of the current lightweight model from various perspectives. Due to the complex structure and many branches of the MobileNetV3 model, which is not GPU friendly, the GPU inference speed is not as good as that of MobileNetV1. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| MobileNetV1_x0_25 | 0.514 | 0.755 | 0.506 | | 0.070 | 0.460 | +| MobileNetV1_x0_5 | 0.635 | 0.847 | 0.637 | | 0.280 | 1.310 | +| MobileNetV1_x0_75 | 0.688 | 0.882 | 0.684 | | 0.630 | 2.550 | +| MobileNetV1 | 0.710 | 0.897 | 0.706 | | 1.110 | 4.190 | +| MobileNetV1_ssld | 0.779 | 0.939 | | | 1.110 | 4.190 | +| MobileNetV2_x0_25 | 0.532 | 0.765 | | | 0.050 | 1.500 | +| MobileNetV2_x0_5 | 0.650 | 0.857 | 0.654 | 0.864 | 0.170 | 1.930 | +| MobileNetV2_x0_75 | 0.698 | 0.890 | 0.698 | 0.896 | 0.350 | 2.580 | +| MobileNetV2 | 0.722 | 0.907 | 0.718 | 0.910 | 0.600 | 3.440 | +| MobileNetV2_x1_5 | 0.741 | 0.917 | | | 1.320 | 6.760 | +| MobileNetV2_x2_0 | 0.752 | 0.926 | | | 2.320 | 11.130 | +| MobileNetV2_ssld | 0.7674 | 0.9339 | | | 0.600 | 3.440 | +| MobileNetV3_large_
x1_25 | 0.764 | 0.930 | 0.766 | | 0.714 | 7.440 | +| MobileNetV3_large_
x1_0 | 0.753 | 0.923 | 0.752 | | 0.450 | 5.470 | +| MobileNetV3_large_
x0_75 | 0.731 | 0.911 | 0.733 | | 0.296 | 3.910 | +| MobileNetV3_large_
x0_5 | 0.692 | 0.885 | 0.688 | | 0.138 | 2.670 | +| MobileNetV3_large_
x0_35 | 0.643 | 0.855 | 0.642 | | 0.077 | 2.100 | +| MobileNetV3_small_
x1_25 | 0.707 | 0.895 | 0.704 | | 0.195 | 3.620 | +| MobileNetV3_small_
x1_0 | 0.682 | 0.881 | 0.675 | | 0.123 | 2.940 | +| MobileNetV3_small_
x0_75 | 0.660 | 0.863 | 0.654 | | 0.088 | 2.370 | +| MobileNetV3_small_
x0_5 | 0.592 | 0.815 | 0.580 | | 0.043 | 1.900 | +| MobileNetV3_small_
x0_35 | 0.530 | 0.764 | 0.498 | | 0.026 | 1.660 | +| MobileNetV3_small_
x0_35_ssld | 0.556 | 0.777 | 0.498 | | 0.026 | 1.660 | +| MobileNetV3_large_
x1_0_ssld | 0.790 | 0.945 | | | 0.450 | 5.470 | +| MobileNetV3_large_
x1_0_ssld_int8 | 0.761 | | | | | | +| MobileNetV3_small_
x1_0_ssld | 0.713 | 0.901 | | | 0.123 | 2.940 | +| ShuffleNetV2 | 0.688 | 0.885 | 0.694 | | 0.280 | 2.260 | +| ShuffleNetV2_x0_25 | 0.499 | 0.738 | | | 0.030 | 0.600 | +| ShuffleNetV2_x0_33 | 0.537 | 0.771 | | | 0.040 | 0.640 | +| ShuffleNetV2_x0_5 | 0.603 | 0.823 | 0.603 | | 0.080 | 1.360 | +| ShuffleNetV2_x1_5 | 0.716 | 0.902 | 0.726 | | 0.580 | 3.470 | +| ShuffleNetV2_x2_0 | 0.732 | 0.912 | 0.749 | | 1.120 | 7.320 | +| ShuffleNetV2_swish | 0.700 | 0.892 | | | 0.290 | 2.260 | +| GhostNet_x0_5 | 0.668 | 0.869 | 0.662 | 0.866 | 0.082 | 2.600 | +| GhostNet_x1_0 | 0.740 | 0.916 | 0.739 | 0.914 | 0.294 | 5.200 | +| GhostNet_x1_3 | 0.757 | 0.925 | 0.757 | 0.927 | 0.440 | 7.300 | +| GhostNet_x1_3_ssld | 0.794 | 0.945 | 0.757 | 0.927 | 0.440 | 7.300 | + + +## 3. Inference speed and storage size based on SD855 + +| Models | Batch Size=1(ms) | Storage Size(M) | +|:--:|:--:|:--:| +| MobileNetV1_x0_25 | 3.220 | 1.900 | +| MobileNetV1_x0_5 | 9.580 | 5.200 | +| MobileNetV1_x0_75 | 19.436 | 10.000 | +| MobileNetV1 | 32.523 | 16.000 | +| MobileNetV1_ssld | 32.523 | 16.000 | +| MobileNetV2_x0_25 | 3.799 | 6.100 | +| MobileNetV2_x0_5 | 8.702 | 7.800 | +| MobileNetV2_x0_75 | 15.531 | 10.000 | +| MobileNetV2 | 23.318 | 14.000 | +| MobileNetV2_x1_5 | 45.624 | 26.000 | +| MobileNetV2_x2_0 | 74.292 | 43.000 | +| MobileNetV2_ssld | 23.318 | 14.000 | +| MobileNetV3_large_x1_25 | 28.218 | 29.000 | +| MobileNetV3_large_x1_0 | 19.308 | 21.000 | +| MobileNetV3_large_x0_75 | 13.565 | 16.000 | +| MobileNetV3_large_x0_5 | 7.493 | 11.000 | +| MobileNetV3_large_x0_35 | 5.137 | 8.600 | +| MobileNetV3_small_x1_25 | 9.275 | 14.000 | +| MobileNetV3_small_x1_0 | 6.546 | 12.000 | +| MobileNetV3_small_x0_75 | 5.284 | 9.600 | +| MobileNetV3_small_x0_5 | 3.352 | 7.800 | +| MobileNetV3_small_x0_35 | 2.635 | 6.900 | +| MobileNetV3_small_x0_35_ssld | 2.635 | 6.900 | +| MobileNetV3_large_x1_0_ssld | 19.308 | 21.000 | +| MobileNetV3_large_x1_0_ssld_int8 | 14.395 | 10.000 | +| MobileNetV3_small_x1_0_ssld | 6.546 | 12.000 | +| ShuffleNetV2 | 10.941 | 9.000 | +| ShuffleNetV2_x0_25 | 2.329 | 2.700 | +| ShuffleNetV2_x0_33 | 2.643 | 2.800 | +| ShuffleNetV2_x0_5 | 4.261 | 5.600 | +| ShuffleNetV2_x1_5 | 19.352 | 14.000 | +| ShuffleNetV2_x2_0 | 34.770 | 28.000 | +| ShuffleNetV2_swish | 16.023 | 9.100 | +| GhostNet_x0_5 | 5.714 | 10.000 | +| GhostNet_x1_0 | 13.558 | 20.000 | +| GhostNet_x1_3 | 19.982 | 29.000 | +| GhostNet_x1_3_ssld | 19.982 | 29.000 | + + +## 4. Inference speed based on T4 GPU + +| Models | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-----------------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------| +| MobileNetV1_x0_25 | 0.68422 | 1.13021 | 1.72095 | 0.67274 | 1.226 | 1.84096 | +| MobileNetV1_x0_5 | 0.69326 | 1.09027 | 1.84746 | 0.69947 | 1.43045 | 2.39353 | +| MobileNetV1_x0_75 | 0.6793 | 1.29524 | 2.15495 | 0.79844 | 1.86205 | 3.064 | +| MobileNetV1 | 0.71942 | 1.45018 | 2.47953 | 0.91164 | 2.26871 | 3.90797 | +| MobileNetV1_ssld | 0.71942 | 1.45018 | 2.47953 | 0.91164 | 2.26871 | 3.90797 | +| MobileNetV2_x0_25 | 2.85399 | 3.62405 | 4.29952 | 2.81989 | 3.52695 | 4.2432 | +| MobileNetV2_x0_5 | 2.84258 | 3.1511 | 4.10267 | 2.80264 | 3.65284 | 4.31737 | +| MobileNetV2_x0_75 | 2.82183 | 3.27622 | 4.98161 | 2.86538 | 3.55198 | 5.10678 | +| MobileNetV2 | 2.78603 | 3.71982 | 6.27879 | 2.62398 | 3.54429 | 6.41178 | +| MobileNetV2_x1_5 | 2.81852 | 4.87434 | 8.97934 | 2.79398 | 5.30149 | 9.30899 | +| MobileNetV2_x2_0 | 3.65197 | 6.32329 | 11.644 | 3.29788 | 7.08644 | 12.45375 | +| MobileNetV2_ssld | 2.78603 | 3.71982 | 6.27879 | 2.62398 | 3.54429 | 6.41178 | +| MobileNetV3_large_x1_25 | 2.34387 | 3.16103 | 4.79742 | 2.35117 | 3.44903 | 5.45658 | +| MobileNetV3_large_x1_0 | 2.20149 | 3.08423 | 4.07779 | 2.04296 | 2.9322 | 4.53184 | +| MobileNetV3_large_x0_75 | 2.1058 | 2.61426 | 3.61021 | 2.0006 | 2.56987 | 3.78005 | +| MobileNetV3_large_x0_5 | 2.06934 | 2.77341 | 3.35313 | 2.11199 | 2.88172 | 3.19029 | +| MobileNetV3_large_x0_35 | 2.14965 | 2.7868 | 3.36145 | 1.9041 | 2.62951 | 3.26036 | +| MobileNetV3_small_x1_25 | 2.06817 | 2.90193 | 3.5245 | 2.02916 | 2.91866 | 3.34528 | +| MobileNetV3_small_x1_0 | 1.73933 | 2.59478 | 3.40276 | 1.74527 | 2.63565 | 3.28124 | +| MobileNetV3_small_x0_75 | 1.80617 | 2.64646 | 3.24513 | 1.93697 | 2.64285 | 3.32797 | +| MobileNetV3_small_x0_5 | 1.95001 | 2.74014 | 3.39485 | 1.88406 | 2.99601 | 3.3908 | +| MobileNetV3_small_x0_35 | 2.10683 | 2.94267 | 3.44254 | 1.94427 | 2.94116 | 3.41082 | +| MobileNetV3_small_x0_35_ssld | 2.10683 | 2.94267 | 3.44254 | 1.94427 | 2.94116 | 3.41082 | +| MobileNetV3_large_x1_0_ssld | 2.20149 | 3.08423 | 4.07779 | 2.04296 | 2.9322 | 4.53184 | +| MobileNetV3_small_x1_0_ssld | 1.73933 | 2.59478 | 3.40276 | 1.74527 | 2.63565 | 3.28124 | +| ShuffleNetV2 | 1.95064 | 2.15928 | 2.97169 | 1.89436 | 2.26339 | 3.17615 | +| ShuffleNetV2_x0_25 | 1.43242 | 2.38172 | 2.96768 | 1.48698 | 2.29085 | 2.90284 | +| ShuffleNetV2_x0_33 | 1.69008 | 2.65706 | 2.97373 | 1.75526 | 2.85557 | 3.09688 | +| ShuffleNetV2_x0_5 | 1.48073 | 2.28174 | 2.85436 | 1.59055 | 2.18708 | 3.09141 | +| ShuffleNetV2_x1_5 | 1.51054 | 2.4565 | 3.41738 | 1.45389 | 2.5203 | 3.99872 | +| ShuffleNetV2_x2_0 | 1.95616 | 2.44751 | 4.19173 | 2.15654 | 3.18247 | 5.46893 | +| ShuffleNetV2_swish | 2.50213 | 2.92881 | 3.474 | 2.5129 | 2.97422 | 3.69357 | +| GhostNet_x0_5 | 2.64492 | 3.48473 | 4.48844 | 2.36115 | 3.52802 | 3.89444 | +| GhostNet_x1_0 | 2.63120 | 3.92065 | 4.48296 | 2.57042 | 3.56296 | 4.85524 | +| GhostNet_x1_3 | 2.89715 | 3.80329 | 4.81661 | 2.81810 | 3.72071 | 5.92269 | diff --git a/src/PaddleClas/docs/en/models/Others_en.md b/src/PaddleClas/docs/en/models/Others_en.md new file mode 100644 index 0000000..37bb659 --- /dev/null +++ b/src/PaddleClas/docs/en/models/Others_en.md @@ -0,0 +1,64 @@ +# Other networks +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed and storage size based on SD855](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +In 2012, AlexNet network proposed by Alex et al. won the ImageNet competition by far surpassing the second place, and the convolutional neural network and even deep learning attracted wide attention. AlexNet used relu as the activation function of CNN to solve the gradient dispersion problem of sigmoid when the network is deep. During the training, Dropout was used to randomly lose a part of the neurons, avoiding the overfitting of the model. In the network, overlapping maximum pooling is used to replace the average pooling commonly used in CNN, which avoids the fuzzy effect of average pooling and improves the feature richness. In a sense, AlexNet has exploded the research and application of neural networks. + +SqueezeNet achieved the same precision as AlexNet on Imagenet-1k, but only with 1/50 parameters. The core of the network is the Fire module, which used the convolution of 1x1 to achieve channel dimensionality reduction, thus greatly saving the number of parameters. The author created SqueezeNet by stacking a large number of Fire modules. + +VGG is a convolutional neural network developed by researchers at Oxford University's Visual Geometry Group and DeepMind. The network explores the relationship between the depth of the convolutional neural network and its performance. By repeatedly stacking the small convolutional kernel of 3x3 and the maximum pooling layer of 2x2, the multi-layer convolutional neural network is successfully constructed and has achieved good convergence accuracy. In the end, VGG won the runner-up of ILSVRC 2014 classification and the champion of positioning. + +DarkNet53 is designed for object detection by YOLO author in the paper. The network is basically composed of 1x1 and 3x3 kernel, with a total of 53 layers, named DarkNet53. + + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| AlexNet | 0.567 | 0.792 | 0.5720 | | 1.370 | 61.090 | +| SqueezeNet1_0 | 0.596 | 0.817 | 0.575 | | 1.550 | 1.240 | +| SqueezeNet1_1 | 0.601 | 0.819 | | | 0.690 | 1.230 | +| VGG11 | 0.693 | 0.891 | | | 15.090 | 132.850 | +| VGG13 | 0.700 | 0.894 | | | 22.480 | 133.030 | +| VGG16 | 0.720 | 0.907 | 0.715 | 0.901 | 30.810 | 138.340 | +| VGG19 | 0.726 | 0.909 | | | 39.130 | 143.650 | +| DarkNet53 | 0.780 | 0.941 | 0.772 | 0.938 | 18.580 | 41.600 | + + + +## 3. Inference speed based on V100 GPU + + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | +|---------------------------|-----------|-------------------|----------------------| +| AlexNet | 224 | 256 | 1.176 | +| SqueezeNet1_0 | 224 | 256 | 0.860 | +| SqueezeNet1_1 | 224 | 256 | 0.763 | +| VGG11 | 224 | 256 | 1.867 | +| VGG13 | 224 | 256 | 2.148 | +| VGG16 | 224 | 256 | 2.616 | +| VGG19 | 224 | 256 | 3.076 | +| DarkNet53 | 256 | 256 | 3.139 | + + +## 4. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-----------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| AlexNet | 224 | 256 | 1.06447 | 1.70435 | 2.38402 | 1.44993 | 2.46696 | 3.72085 | +| SqueezeNet1_0 | 224 | 256 | 0.97162 | 2.06719 | 3.67499 | 0.96736 | 2.53221 | 4.54047 | +| SqueezeNet1_1 | 224 | 256 | 0.81378 | 1.62919 | 2.68044 | 0.76032 | 1.877 | 3.15298 | +| VGG11 | 224 | 256 | 2.24408 | 4.67794 | 7.6568 | 3.90412 | 9.51147 | 17.14168 | +| VGG13 | 224 | 256 | 2.58589 | 5.82708 | 10.03591 | 4.64684 | 12.61558 | 23.70015 | +| VGG16 | 224 | 256 | 3.13237 | 7.19257 | 12.50913 | 5.61769 | 16.40064 | 32.03939 | +| VGG19 | 224 | 256 | 3.69987 | 8.59168 | 15.07866 | 6.65221 | 20.4334 | 41.55902 | +| DarkNet53 | 256 | 256 | 3.18101 | 5.88419 | 10.14964 | 4.10829 | 12.1714 | 22.15266 | diff --git a/src/PaddleClas/docs/en/models/PP-LCNet_en.md b/src/PaddleClas/docs/en/models/PP-LCNet_en.md new file mode 100644 index 0000000..1dd35e0 --- /dev/null +++ b/src/PaddleClas/docs/en/models/PP-LCNet_en.md @@ -0,0 +1,169 @@ +# PP-LCNet Series +--- + + +## Catalogue + +- [1. Abstract](#1) +- [2. Introduction](#2) +- [3. Method](#3) + - [3.1 Better Activation Function](#3.1) + - [3.2 SE Modules at Appropriate Positions](#3.2) + - [3.3 Larger Convolution Kernels](#3.3) + - [3.4 Larger Dimensional 1 × 1 Conv Layer after GAP](#3.4) +- [4. Experiments](#4) + - [4.1 Image Classification](#4.1) + - [4.2 Object Detection](#4.2) + - [4.3 Semantic Segmentation](#4.3) +- [5. Conclusion](#5) +- [6. Reference](#6) + + +## 1. Abstract + +In the field of computer vision, the quality of backbone network determines the outcome of the whole vision task. In previous studies, researchers generally focus on the optimization of FLOPs or Params, but inference speed actually serves as an importance indicator of model quality in real-world scenarios. Nevertheless, it is difficult to balance inference speed and accuracy. In view of various CPU-based applications in industry, we are now working to raise the adaptability of the backbone network to Intel CPU, so as to obtain a faster and more accurate lightweight backbone network. At the same time, the performance of downstream vision tasks such as object detection and semantic segmentation are also improved. + + +## 2. Introduction + +Recent years witnessed the emergence of many lightweight backbone networks. In past two years, in particular, there were abundant networks searched by NAS that either enjoy advantages on FLOPs or Params, or have an edge in terms of inference speed on ARM devices. However, few of them dedicated to specified optimization of Intel CPU, resulting their imperfect inference speed on the intel CPU side. Based on this, we specially design the backbone network PP-LCNet for Intel CPU devices with its acceleration library MKLDNN. Compared with other lightweight SOTA models, this backbone network can further improve the performance of the model without increasing the inference time, significantly outperforming the existing SOTA models. A comparison chart with other models is shown below. +![](../../images/PP-LCNet/PP-LCNet-Acc.png) + + +## 3. Method + +The overall structure of the network is shown in the figure below. +![](../../images/PP-LCNet/PP-LCNet.png) + +Build on extensive experiments, we found that many seemingly less time-consuming operations will increase the latency on Intel CPU-based devices, especially when the MKLDNN acceleration library is enabled. Therefore, we finally chose a block with the leanest possible structure and the fastest possible speed to form our BaseNet (similar to MobileNetV1). Based on BaseNet, we summarized four strategies that can improve the accuracy of the model without increasing the latency, and we combined these four strategies to form PP-LCNet. Each of these four strategies is introduced as below: + + +### 3.1 Better Activation Function + +Since the adoption of ReLU activation function by convolutional neural network, the network performance has been improved substantially, and variants of the ReLU activation function have appeared in recent years, such as Leaky-ReLU, P-ReLU, ELU, etc. In 2017, Google Brain searched to obtain the swish activation function, which performs well on lightweight networks. In 2019, the authors of MobileNetV3 further optimized this activation function to H-Swish, which removes the exponential operation, leading to faster speed and an almost unaffected network accuracy. After many experiments, we also recognized its excellent performance on lightweight networks. Therefore, this activation function is adopted in PP-LCNet. + + +### 3.2 SE Modules at Appropriate Positions + +The SE module is a channel attention mechanism proposed by SENet, which can effectively improve the accuracy of the model. However, on the Intel CPU side, the module also presents a large latency, leaving us the task of balancing accuracy and speed. The search of the location of the SE module in NAS search-based networks such as MobileNetV3 brings no general conclusions, but we found through our experiments that the closer the SE module is to the tail of the network the greater the improvement in model accuracy. The following table also shows some of our experimental results： + +| SE Location | Top-1 Acc(\%) | Latency(ms) | +|-------------------|---------------|-------------| +| 1100000000000 | 61.73 | 2.06 | +| 0000001100000 | 62.17 | 2.03 | +| 0000000000011 | 63.14 | 2.05 | +| 1111111111111 | 64.27 | 3.80 | + +The option in the third row of the table was chosen for the location of the SE module in PP-LCNet. + + +### 3.3 Larger Convolution Kernels + +In the paper of MixNet, the author analyzes the effect of convolutional kernel size on model performance and concludes that larger convolutional kernels within a certain range can improve the performance of the model, but beyond this range will be detrimental to the model’s performance. So the author forms MixConv with split-concat paradigm combined, which can improve the performance of the model but is not conducive to inference. We experimentally summarize the role of some larger convolutional kernels at different positions that are similar to those of the SE module, and find that larger convolutional kernels display more prominent roles in the middle and tail of the network. The following table shows the effect of the position of the 5x5 convolutional kernels on the accuracy： + +| Larger Convolution Location | Top-1 Acc(\%) | Latency(ms) | +|----------------------------|---------------|-------------| +| 1111111111111 | 63.22 | 2.08 | +| 1111111000000 | 62.70 | 2.07 | +| 0000001111111 | 63.14 | 2.05 | + + +Experiments show that a larger convolutional kernel placed at the middle and tail of the network can achieve the same accuracy as placed at all positions, coupled with faster inference. The option in the third row of the table was the final choice of PP-LCNet. + + +### 3.4 Larger Dimensional 1 × 1 Conv Layer after GAP + +Since the introduction of GoogLeNet, GAP (Global-Average-Pooling) is often directly followed by a classification layer, which fails to result in further integration and processing of features extracted after GAP in the lightweight network. If a larger 1x1 convolutional layer (equivalent to the FC layer) is used after GAP, the extracted features, instead of directly passing through the classification layer, will first be integrated, and then classified. This can greatly improve the accuracy rate without affecting the inference speed of the model. The above four improvements were made to BaseNet to obtain PP-LCNet. The following table further illustrates the impact of each scheme on the results： + +| Activation | SE-block | Large-kernal | last-1x1-conv | Top-1 Acc(\%) | Latency(ms) | +|------------|----------|--------------|---------------|---------------|-------------| +| 0 | 1 | 1 | 1 | 61.93 | 1.94 | +| 1 | 0 | 1 | 1 | 62.51 | 1.87 | +| 1 | 1 | 0 | 1 | 62.44 | 2.01 | +| 1 | 1 | 1 | 0 | 59.91 | 1.85 | +| 1 | 1 | 1 | 1 | 63.14 | 2.05 | + + +## 4. Experiments + + +### 4.1 Image Classification + +For image classification, ImageNet dataset is adopted. Compared with the current mainstream lightweight network, PP-LCNet can obtain faster inference speed with the same accuracy. When using Baidu’s self-developed SSLD distillation strategy, the accuracy is further improved, with the Top-1 Acc of ImageNet exceeding 80% at an inference speed of about 5ms on the Intel CPU side. + +| Model | Params(M) | FLOPs(M) | Top-1 Acc(\%) | Top-5 Acc(\%) | Latency(ms) | +|-------|-----------|----------|---------------|---------------|-------------| +| PP-LCNet-0.25x | 1.5 | 18 | 51.86 | 75.65 | 1.74 | +| PP-LCNet-0.35x | 1.6 | 29 | 58.09 | 80.83 | 1.92 | +| PP-LCNet-0.5x | 1.9 | 47 | 63.14 | 84.66 | 2.05 | +| PP-LCNet-0.75x | 2.4 | 99 | 68.18 | 88.30 | 2.29 | +| PP-LCNet-1x | 3.0 | 161 | 71.32 | 90.03 | 2.46 | +| PP-LCNet-1.5x | 4.5 | 342 | 73.71 | 91.53 | 3.19 | +| PP-LCNet-2x | 6.5 | 590 | 75.18 | 92.27 | 4.27 | +| PP-LCNet-2.5x | 9.0 | 906 | 76.60 | 93.00 | 5.39 | +| PP-LCNet-0.5x\* | 1.9 | 47 | 66.10 | 86.46 | 2.05 | +| PP-LCNet-1.0x\* | 3.0 | 161 | 74.39 | 92.09 | 2.46 | +| PP-LCNet-2.5x\* | 9.0 | 906 | 80.82 | 95.33 | 5.39 | + +\* denotes the model after using SSLD distillation. + +Performance comparison with other lightweight networks: + +| Model | Params(M) | FLOPs(M) | Top-1 Acc(\%) | Top-5 Acc(\%) | Latency(ms) | +|-------|-----------|----------|---------------|---------------|-------------| +| MobileNetV2-0.25x | 1.5 | 34 | 53.21 | 76.52 | 2.47 | +| MobileNetV3-small-0.35x | 1.7 | 15 | 53.03 | 76.37 | 3.02 | +| ShuffleNetV2-0.33x | 0.6 | 24 | 53.73 | 77.05 | 4.30 | +| PP-LCNet-0.25x | 1.5 | 18 | 51.86 | 75.65 | 1.74 | +| MobileNetV2-0.5x | 2.0 | 99 | 65.03 | 85.72 | 2.85 | +| MobileNetV3-large-0.35x | 2.1 | 41 | 64.32 | 85.46 | 3.68 | +| ShuffleNetV2-0.5x | 1.4 | 43 | 60.32 | 82.26 | 4.65 | +| PP-LCNet-0.5x | 1.9 | 47 | 63.14 | 84.66 | 2.05 | +| MobileNetV1-1x | 4.3 | 578 | 70.99 | 89.68 | 3.38 | +| MobileNetV2-1x | 3.5 | 327 | 72.15 | 90.65 | 4.26 | +| MobileNetV3-small-1.25x | 3.6 | 100 | 70.67 | 89.51 | 3.95 | +| PP-LCNet-1x | 3.0 | 161 | 71.32 | 90.03 | 2.46 | + + +### 4.2 Object Detection + +For object detection, we adopt Baidu’s self-developed PicoDet, which focuses on lightweight object detection scenarios. The following table shows the comparison between the results of PP-LCNet and MobileNetV3 on the COCO dataset. PP-LCNet has an obvious advantage in both accuracy and speed. + +| Backbone | mAP(%) | Latency(ms) | +|-------|-----------|----------| +MobileNetV3-large-0.35x | 19.2 | 8.1 | +PP-LCNet-0.5x | 20.3 | 6.0 | +MobileNetV3-large-0.75x | 25.8 | 11.1 | +PP-LCNet-1x | 26.9 | 7.9 | + + +### 4.3 Semantic Segmentation + +For semantic segmentation, DeeplabV3+ is adopted. The following table presents the comparison between PP-LCNet and MobileNetV3 on the Cityscapes dataset, and PP-LCNet also stands out in terms of accuracy and speed. + +| Backbone | mIoU(%) | Latency(ms) | +|-------|-----------|----------| +MobileNetV3-large-0.5x | 55.42 | 135 | +PP-LCNet-0.5x | 58.36 | 82 | +MobileNetV3-large-0.75x | 64.53 | 151 | +PP-LCNet-1x | 66.03 | 96 | + + +## 5. Conclusion + +Rather than holding on to perfect FLOPs and Params as academics do, PP-LCNet focuses on analyzing how to add Intel CPU-friendly modules to improve the performance of the model, which can better balance accuracy and inference time. The experimental conclusions therein are available to other researchers in network structure design, while providing NAS search researchers with a smaller search space and general conclusions. The finished PP-LCNet can also be better accepted and applied in industry. + + +## 6. Reference + +Reference to cite when you use PP-LCNet in a paper: +``` +@misc{cui2021pplcnet, + title={PP-LCNet: A Lightweight CPU Convolutional Neural Network}, + author={Cheng Cui and Tingquan Gao and Shengyu Wei and Yuning Du and Ruoyu Guo and Shuilong Dong and Bin Lu and Ying Zhou and Xueying Lv and Qiwen Liu and Xiaoguang Hu and Dianhai Yu and Yanjun Ma}, + year={2021}, + eprint={2109.15099}, + archivePrefix={arXiv}, + primaryClass={cs.CV} +} +``` diff --git a/src/PaddleClas/docs/en/models/PVTV2_en.md b/src/PaddleClas/docs/en/models/PVTV2_en.md new file mode 100644 index 0000000..39b9b8a --- /dev/null +++ b/src/PaddleClas/docs/en/models/PVTV2_en.md @@ -0,0 +1,26 @@ +# PVTV2 + +--- + +## Content + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +PVTV2 is VisionTransformer series model, which build on PVT (Pyramid Vision Transformer). PVT use Transformer block to build feature pyramid network. The mainly designs of PVTV2 are: (1) overlapping patch embedding, (2) convolutional feedforward networks, and (3) linear complexity attention layers. [Paper](https://arxiv.org/pdf/2106.13797.pdf). + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| PVT_V2_B0 | 0.705 | 0.902 | 0.705 | - | 0.53 | 3.7 | +| PVT_V2_B1 | 0.787 | 0.945 | 0.787 | - | 2.0 | 14.0 | +| PVT_V2_B2 | 0.821 | 0.960 | 0.820 | - | 3.9 | 25.4 | +| PVT_V2_B3 | 0.831 | 0.965 | 0.831 | - | 6.7 | 45.2 | +| PVT_V2_B4 | 0.836 | 0.967 | 0.836 | - | 9.8 | 62.6 | +| PVT_V2_B5 | 0.837 | 0.966 | 0.838 | - | 11.4 | 82.0 | +| PVT_V2_B2_Linear | 0.821 | 0.961 | 0.821 | - | 3.8 | 22.6 | diff --git a/src/PaddleClas/docs/en/models/ReXNet_en.md b/src/PaddleClas/docs/en/models/ReXNet_en.md new file mode 100644 index 0000000..63f693c --- /dev/null +++ b/src/PaddleClas/docs/en/models/ReXNet_en.md @@ -0,0 +1,24 @@ +# ReXNet series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## Overview + + ReXNet is proposed by NAVER AI Lab, which is based on new network design principles. Aiming at the problem of representative bottleneck in the existing network, a set of design principles are proposed. The author believes that the conventional design produce representational bottlenecks, which would affect model performance. To investigate the representational bottleneck, the author study the matrix rank of the features generated by ten thousand random networks. Besides, entire layer’s channel configuration is also studied to design more accurate network architectures. In the end, the author proposes a set of simple and effective design principles to mitigate the representational bottleneck. [paper](https://arxiv.org/pdf/2007.00992.pdf) + + +## Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | FLOPs
(G) | Params
(M) | +| :--------: | :---: | :---: | :---------------: | :-----------: | -------------- | +| ReXNet_1_0 | 77.46 | 93.70 | 77.9 | 0.415 | 4.838 | +| ReXNet_1_3 | 79.13 | 94.64 | 79.5 | 0.683 | 7.611 | +| ReXNet_1_5 | 80.06 | 95.12 | 80.3 | 0.900 | 9.791 | +| ReXNet_2_0 | 81.22 | 95.36 | 81.6 | 1.561 | 16.449 | +| ReXNet_3_0 | 82.09 | 96.12 | 82.8 | 3.445 | 34.833 | + +Inference speed and other information are coming soon. diff --git a/src/PaddleClas/docs/en/models/RedNet_en.md b/src/PaddleClas/docs/en/models/RedNet_en.md new file mode 100644 index 0000000..0926837 --- /dev/null +++ b/src/PaddleClas/docs/en/models/RedNet_en.md @@ -0,0 +1,22 @@ +# RedNet series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +In the backbone of ResNet and in all bottleneck positions of backbone, the convolution is replaced by Involution, but all convolutions are reserved for channel mapping and fusion. These carefully redesigned entities combine to form a new efficient backbone network, called Rednet. [paper](https://arxiv.org/abs/2103.06255). + + +## 2. Accuracy, FLOPs and Parameters + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:---------------------:|:----------:|:---------:|:---------:|:---------:| +| RedNet26 | 9.2 | 1.7 | 75.95 | 93.19 | +| RedNet38 | 12.4 | 2.2 | 77.47 | 93.56 | +| RedNet50 | 15.5 | 2.7 | 78.33 | 94.17 | +| RedNet101 | 25.7 | 4.7 | 78.94 | 94.36 | +| RedNet152 | 34.0 | 6.8 | 79.17 | 94.40 | \ No newline at end of file diff --git a/src/PaddleClas/docs/en/models/RepVGG_en.md b/src/PaddleClas/docs/en/models/RepVGG_en.md new file mode 100644 index 0000000..551feaf --- /dev/null +++ b/src/PaddleClas/docs/en/models/RepVGG_en.md @@ -0,0 +1,29 @@ +# RepVGG series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +RepVGG (Making VGG-style ConvNets Great Again) series model is a simple but powerful convolutional neural network architecture proposed by Tsinghua University (Guiguang Ding's team), MEGVII Technology (Jian Sun et al.), HKUST and Aberystwyth University in 2021. The architecture has an inference time agent similar to VGG. The main body is composed of 3x3 convolution and relu stack, while the training time model has multi branch topology. The decoupling of training time and inference time is realized by re-parameterization technology, so the model is called repvgg. [paper](https://arxiv.org/abs/2101.03697). + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1| FLOPs
(G) | +|:--:|:--:|:--:|:--:|:--:| +| RepVGG_A0 | 0.7131 | 0.9016 | 0.7241 | | +| RepVGG_A1 | 0.7380 | 0.9146 | 0.7446 | | +| RepVGG_A2 | 0.7571 | 0.9264 | 0.7648 | | +| RepVGG_B0 | 0.7450 | 0.9213 | 0.7514 | | +| RepVGG_B1 | 0.7773 | 0.9385 | 0.7837 | | +| RepVGG_B2 | 0.7813 | 0.9410 | 0.7878 | | +| RepVGG_B1g2 | 0.7732 | 0.9359 | 0.7778 | | +| RepVGG_B1g4 | 0.7675 | 0.9335 | 0.7758 | | +| RepVGG_B2g4 | 0.7881 | 0.9448 | 0.7938 | | +| RepVGG_B3g4 | 0.7965 | 0.9485 | 0.8021 | | + +Params, FLOPs, Inference speed and other information are coming soon. diff --git a/src/PaddleClas/docs/en/models/ResNeSt_RegNet_en.md b/src/PaddleClas/docs/en/models/ResNeSt_RegNet_en.md new file mode 100644 index 0000000..b6ce389 --- /dev/null +++ b/src/PaddleClas/docs/en/models/ResNeSt_RegNet_en.md @@ -0,0 +1,32 @@ +# ResNeSt and RegNet series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed based on T4 GPU](#3) + + +## 1. Overview + +The ResNeSt series was proposed in 2020. The original resnet network structure has been improved by introducing K groups and adding an attention module similar to SEBlock in different groups, the accuracy is greater than that of the basic model ResNet, but the parameter amount and flops are almost the same as the basic ResNet. + +RegNet was proposed in 2020 by Facebook to deepen the concept of design space. Based on AnyNetX, the model performance is gradually improved by shared bottleneck ratio, shared group width, adjusting network depth or width and other strategies. What's more, the design space structure is simplified, whose interpretability is also be improved. The quality of design space is improved while its diversity is maintained. Under similar conditions, the performance of the designed RegNet model performs better than EfficientNet and 5 times faster than EfficientNet. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ResNeSt50_fast_1s1x64d | 0.8035 | 0.9528| 0.8035 | -| 8.68 | 26.3 | +| ResNeSt50 | 0.8083 | 0.9542| 0.8113 | -| 10.78 | 27.5 | +| RegNetX_4GF | 0.7850 | 0.9416| 0.7860 | -| 8.0 | 22.1 | + + +## 3. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|--------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| ResNeSt50_fast_1s1x64d | 224 | 256 | 3.46466 | 5.56647 | 9.11848 | 3.45405 | 8.72680 | 15.48710 | +| ResNeSt50 | 224 | 256 | 7.05851 | 8.97676 | 13.34704 | 6.16248 | 12.0633 | 21.49936 | +| RegNetX_4GF | 224 | 256 | 6.69042 | 8.01664 | 11.60608 | 6.46478 | 11.19862 | 16.89089 | diff --git a/src/PaddleClas/docs/en/models/ResNet_and_vd_en.md b/src/PaddleClas/docs/en/models/ResNet_and_vd_en.md new file mode 100644 index 0000000..70312fd --- /dev/null +++ b/src/PaddleClas/docs/en/models/ResNet_and_vd_en.md @@ -0,0 +1,104 @@ +# ResNet and ResNet_vd series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed based on V100 GPU](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +The ResNet series model was proposed in 2015 and won the championship in the ILSVRC2015 competition with a top5 error rate of 3.57%. The network innovatively proposed the residual structure, and built the ResNet network by stacking multiple residual structures. Experiments show that using residual blocks can improve the convergence speed and accuracy effectively. + +Joyce Xu of Stanford university calls ResNet one of three architectures that "really redefine the way we think about neural networks." Due to the outstanding performance of ResNet, more and more scholars and engineers from academia and industry have improved its structure. The well-known ones include wide-resnet, resnet-vc, resnet-vd, Res2Net, etc. The number of parameters and FLOPs of resnet-vc and resnet-vd are almost the same as those of ResNet, so we hereby unified them into the ResNet series. + +The models of the ResNet series released this time include 14 pre-trained models including ResNet50, ResNet50_vd, ResNet50_vd_ssld, and ResNet200_vd. At the training level, ResNet adopted the standard training process for training ImageNet, while the rest of the improved model adopted more training strategies, such as cosine decay for the decline of learning rate and the regular label smoothing method,mixup was added to the data preprocessing, and the total number of iterations increased from 120 epoches to 200 epoches. + +Among them, ResNet50_vd_v2 and ResNet50_vd_ssld adopted knowledge distillation, which further improved the accuracy of the model while keeping the structure unchanged. Specifically, the teacher model of ResNet50_vd_v2 is ResNet152_vd (top1 accuracy 80.59%), the training set is imagenet-1k, the teacher model of ResNet50_vd_ssld is ResNeXt101_32x16d_wsl (top1 accuracy 84.2%), and the training set is the combination of 4 million data mined by imagenet-22k and ImageNet-1k . The specific methods of knowledge distillation are being continuously updated. + +The FLOPs, parameters, and inference time on the T4 GPU of this series of models are shown in the figure below. + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.ResNet.png) + + +As can be seen from the above curves, the higher the number of layers, the higher the accuracy, but the corresponding number of parameters, calculation and latency will increase. ResNet50_vd_ssld further improves the accuracy of top-1 of the ImageNet-1k validation set by using stronger teachers and more data, reaching 82.39%, refreshing the accuracy of ResNet50 series models. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ResNet18 | 0.710 | 0.899 | 0.696 | 0.891 | 3.660 | 11.690 | +| ResNet18_vd | 0.723 | 0.908 | | | 4.140 | 11.710 | +| ResNet34 | 0.746 | 0.921 | 0.732 | 0.913 | 7.360 | 21.800 | +| ResNet34_vd | 0.760 | 0.930 | | | 7.390 | 21.820 | +| ResNet34_vd_ssld | 0.797 | 0.949 | | | 7.390 | 21.820 | +| ResNet50 | 0.765 | 0.930 | 0.760 | 0.930 | 8.190 | 25.560 | +| ResNet50_vc | 0.784 | 0.940 | | | 8.670 | 25.580 | +| ResNet50_vd | 0.791 | 0.944 | 0.792 | 0.946 | 8.670 | 25.580 | +| ResNet50_vd_v2 | 0.798 | 0.949 | | | 8.670 | 25.580 | +| ResNet101 | 0.776 | 0.936 | 0.776 | 0.938 | 15.520 | 44.550 | +| ResNet101_vd | 0.802 | 0.950 | | | 16.100 | 44.570 | +| ResNet152 | 0.783 | 0.940 | 0.778 | 0.938 | 23.050 | 60.190 | +| ResNet152_vd | 0.806 | 0.953 | | | 23.530 | 60.210 | +| ResNet200_vd | 0.809 | 0.953 | | | 30.530 | 74.740 | +| ResNet50_vd_ssld | 0.824 | 0.961 | | | 8.670 | 25.580 | +| ResNet50_vd_ssld_v2 | 0.830 | 0.964 | | | 8.670 | 25.580 | +| Fix_ResNet50_vd_ssld_v2 | 0.840 | 0.970 | | | 17.696 | 25.580 | +| ResNet101_vd_ssld | 0.837 | 0.967 | | | 16.100 | 44.570 | + +* Note: `ResNet50_vd_ssld_v2` is obtained by adding AutoAugment in training process on the basis of `ResNet50_vd_ssld` training strategy.`Fix_ResNet50_vd_ssld_v2` stopped all parameter updates of `ResNet50_vd_ssld_v2` except the FC layer,and fine-tuned on ImageNet1k dataset, the resolution is 320x320. + + +## 3. Inference speed based on V100 GPU + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | +|------------------|-----------|-------------------|--------------------------| +| ResNet18 | 224 | 256 | 1.499 | +| ResNet18_vd | 224 | 256 | 1.603 | +| ResNet34 | 224 | 256 | 2.272 | +| ResNet34_vd | 224 | 256 | 2.343 | +| ResNet34_vd_ssld | 224 | 256 | 2.343 | +| ResNet50 | 224 | 256 | 2.939 | +| ResNet50_vc | 224 | 256 | 3.041 | +| ResNet50_vd | 224 | 256 | 3.165 | +| ResNet50_vd_v2 | 224 | 256 | 3.165 | +| ResNet101 | 224 | 256 | 5.314 | +| ResNet101_vd | 224 | 256 | 5.252 | +| ResNet152 | 224 | 256 | 7.205 | +| ResNet152_vd | 224 | 256 | 7.200 | +| ResNet200_vd | 224 | 256 | 8.885 | +| ResNet50_vd_ssld | 224 | 256 | 3.165 | +| ResNet101_vd_ssld | 224 | 256 | 5.252 | + + +## 4. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| ResNet18 | 224 | 256 | 1.3568 | 2.5225 | 3.61904 | 1.45606 | 3.56305 | 6.28798 | +| ResNet18_vd | 224 | 256 | 1.39593 | 2.69063 | 3.88267 | 1.54557 | 3.85363 | 6.88121 | +| ResNet34 | 224 | 256 | 2.23092 | 4.10205 | 5.54904 | 2.34957 | 5.89821 | 10.73451 | +| ResNet34_vd | 224 | 256 | 2.23992 | 4.22246 | 5.79534 | 2.43427 | 6.22257 | 11.44906 | +| ResNet34_vd | 224 | 256 | 2.23992 | 4.22246 | 5.79534 | 2.43427 | 6.22257 | 11.44906 | +| ResNet50 | 224 | 256 | 2.63824 | 4.63802 | 7.02444 | 3.47712 | 7.84421 | 13.90633 | +| ResNet50_vc | 224 | 256 | 2.67064 | 4.72372 | 7.17204 | 3.52346 | 8.10725 | 14.45577 | +| ResNet50_vd | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| ResNet50_vd_v2 | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| ResNet101 | 224 | 256 | 5.04037 | 7.73673 | 10.8936 | 6.07125 | 13.40573 | 24.3597 | +| ResNet101_vd | 224 | 256 | 5.05972 | 7.83685 | 11.34235 | 6.11704 | 13.76222 | 25.11071 | +| ResNet152 | 224 | 256 | 7.28665 | 10.62001 | 14.90317 | 8.50198 | 19.17073 | 35.78384 | +| ResNet152_vd | 224 | 256 | 7.29127 | 10.86137 | 15.32444 | 8.54376 | 19.52157 | 36.64445 | +| ResNet200_vd | 224 | 256 | 9.36026 | 13.5474 | 19.0725 | 10.80619 | 25.01731 | 48.81399 | +| ResNet50_vd_ssld | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| ResNet50_vd_ssld_v2 | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| Fix_ResNet50_vd_ssld_v2 | 320 | 320 | 3.42818 | 7.51534 | 13.19370 | 5.07696 | 14.64218 | 27.01453 | +| ResNet101_vd_ssld | 224 | 256 | 5.05972 | 7.83685 | 11.34235 | 6.11704 | 13.76222 | 25.11071 | diff --git a/src/PaddleClas/docs/en/models/SEResNext_and_Res2Net_en.md b/src/PaddleClas/docs/en/models/SEResNext_and_Res2Net_en.md new file mode 100644 index 0000000..18c8ace --- /dev/null +++ b/src/PaddleClas/docs/en/models/SEResNext_and_Res2Net_en.md @@ -0,0 +1,126 @@ +# SEResNeXt and Res2Net series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) +* [3. Inference speed based on V100 GPU](#3) +* [4. Inference speed based on T4 GPU](#4) + + +## 1. Overview + +ResNeXt, one of the typical variants of ResNet, was presented at the CVPR conference in 2017. Prior to this, the methods to improve the model accuracy mainly focused on deepening or widening the network, which increased the number of parameters and calculation, and slowed down the inference speed accordingly. The concept of cardinality was proposed in ResNeXt structure. The author found that increasing the number of channel groups was more effective than increasing the depth and width through experiments. It can improve the accuracy without increasing the parameter complexity and reduce the number of parameters at the same time, so it is a more successful variant of ResNet. + +SENet is the winner of the 2017 ImageNet classification competition. It proposes a new SE structure that can be migrated to any other network. It controls the scale to enhance the important features between each channel, and weaken the unimportant features. So that the extracted features are more directional. + +Res2Net is a brand-new improvement of ResNet proposed in 2019. The solution can be easily integrated with other excellent modules. Without increasing the amount of calculation, the performance on ImageNet, CIFAR-100 and other data sets exceeds ResNet. Res2Net, with its simple structure and superior performance, further explores the multi-scale representation capability of CNN at a more fine-grained level. Res2Net reveals a new dimension to improve model accuracy, called scale, which is an essential and more effective factor in addition to the existing dimensions of depth, width, and cardinality. The network also performs well in other visual tasks such as object detection and image segmentation. + +The FLOPs, parameters, and inference time on the T4 GPU of this series of models are shown in the figure below. + + +![](../../images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.SeResNeXt.png) + + +At present, there are a total of 24 pretrained models of the three categories open sourced by PaddleClas, and the indicators are shown in the figure. It can be seen from the diagram that under the same Flops and Params, the improved model tends to have higher accuracy, but the inference speed is often inferior to the ResNet series. On the other hand, Res2Net performed better. Compared with group operation in ResNeXt and SE structure operation in SEResNet, Res2Net tended to have better accuracy in the same Flops, Params and inference speed. + + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| Res2Net50_26w_4s | 0.793 | 0.946 | 0.780 | 0.936 | 8.520 | 25.700 | +| Res2Net50_vd_26w_4s | 0.798 | 0.949 | | | 8.370 | 25.060 | +| Res2Net50_vd_26w_4s_ssld | 0.831 | 0.966 | | | 8.370 | 25.060 | +| Res2Net50_14w_8s | 0.795 | 0.947 | 0.781 | 0.939 | 9.010 | 25.720 | +| Res2Net101_vd_26w_4s | 0.806 | 0.952 | | | 16.670 | 45.220 | +| Res2Net101_vd_26w_4s_ssld | 0.839 | 0.971 | | | 16.670 | 45.220 | +| Res2Net200_vd_26w_4s | 0.812 | 0.957 | | | 31.490 | 76.210 | +| Res2Net200_vd_26w_4s_ssld | **0.851** | 0.974 | | | 31.490 | 76.210 | +| ResNeXt50_32x4d | 0.778 | 0.938 | 0.778 | | 8.020 | 23.640 | +| ResNeXt50_vd_32x4d | 0.796 | 0.946 | | | 8.500 | 23.660 | +| ResNeXt50_64x4d | 0.784 | 0.941 | | | 15.060 | 42.360 | +| ResNeXt50_vd_64x4d | 0.801 | 0.949 | | | 15.540 | 42.380 | +| ResNeXt101_32x4d | 0.787 | 0.942 | 0.788 | | 15.010 | 41.540 | +| ResNeXt101_vd_32x4d | 0.803 | 0.951 | | | 15.490 | 41.560 | +| ResNeXt101_64x4d | 0.784 | 0.945 | 0.796 | | 29.050 | 78.120 | +| ResNeXt101_vd_64x4d | 0.808 | 0.952 | | | 29.530 | 78.140 | +| ResNeXt152_32x4d | 0.790 | 0.943 | | | 22.010 | 56.280 | +| ResNeXt152_vd_32x4d | 0.807 | 0.952 | | | 22.490 | 56.300 | +| ResNeXt152_64x4d | 0.795 | 0.947 | | | 43.030 | 107.570 | +| ResNeXt152_vd_64x4d | 0.811 | 0.953 | | | 43.520 | 107.590 | +| SE_ResNet18_vd | 0.733 | 0.914 | | | 4.140 | 11.800 | +| SE_ResNet34_vd | 0.765 | 0.932 | | | 7.840 | 21.980 | +| SE_ResNet50_vd | 0.795 | 0.948 | | | 8.670 | 28.090 | +| SE_ResNeXt50_32x4d | 0.784 | 0.940 | 0.789 | 0.945 | 8.020 | 26.160 | +| SE_ResNeXt50_vd_32x4d | 0.802 | 0.949 | | | 10.760 | 26.280 | +| SE_ResNeXt101_32x4d | 0.7939 | 0.9443 | 0.793 | 0.950 | 15.020 | 46.280 | +| SENet154_vd | 0.814 | 0.955 | | | 45.830 | 114.290 | + + + +## 3. Inference speed based on V100 GPU + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | +|-----------------------|-----------|-------------------|--------------------------| +| Res2Net50_26w_4s | 224 | 256 | 4.148 | +| Res2Net50_vd_26w_4s | 224 | 256 | 4.172 | +| Res2Net50_14w_8s | 224 | 256 | 5.113 | +| Res2Net101_vd_26w_4s | 224 | 256 | 7.327 | +| Res2Net200_vd_26w_4s | 224 | 256 | 12.806 | +| ResNeXt50_32x4d | 224 | 256 | 10.964 | +| ResNeXt50_vd_32x4d | 224 | 256 | 7.566 | +| ResNeXt50_64x4d | 224 | 256 | 13.905 | +| ResNeXt50_vd_64x4d | 224 | 256 | 14.321 | +| ResNeXt101_32x4d | 224 | 256 | 14.915 | +| ResNeXt101_vd_32x4d | 224 | 256 | 14.885 | +| ResNeXt101_64x4d | 224 | 256 | 28.716 | +| ResNeXt101_vd_64x4d | 224 | 256 | 28.398 | +| ResNeXt152_32x4d | 224 | 256 | 22.996 | +| ResNeXt152_vd_32x4d | 224 | 256 | 22.729 | +| ResNeXt152_64x4d | 224 | 256 | 46.705 | +| ResNeXt152_vd_64x4d | 224 | 256 | 46.395 | +| SE_ResNet18_vd | 224 | 256 | 1.694 | +| SE_ResNet34_vd | 224 | 256 | 2.786 | +| SE_ResNet50_vd | 224 | 256 | 3.749 | +| SE_ResNeXt50_32x4d | 224 | 256 | 8.924 | +| SE_ResNeXt50_vd_32x4d | 224 | 256 | 9.011 | +| SE_ResNeXt101_32x4d | 224 | 256 | 19.204 | +| SENet154_vd | 224 | 256 | 50.406 | + + +## 4. Inference speed based on T4 GPU + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-----------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| Res2Net50_26w_4s | 224 | 256 | 3.56067 | 6.61827 | 11.41566 | 4.47188 | 9.65722 | 17.54535 | +| Res2Net50_vd_26w_4s | 224 | 256 | 3.69221 | 6.94419 | 11.92441 | 4.52712 | 9.93247 | 18.16928 | +| Res2Net50_14w_8s | 224 | 256 | 4.45745 | 7.69847 | 12.30935 | 5.4026 | 10.60273 | 18.01234 | +| Res2Net101_vd_26w_4s | 224 | 256 | 6.53122 | 10.81895 | 18.94395 | 8.08729 | 17.31208 | 31.95762 | +| Res2Net200_vd_26w_4s | 224 | 256 | 11.66671 | 18.93953 | 33.19188 | 14.67806 | 32.35032 | 63.65899 | +| ResNeXt50_32x4d | 224 | 256 | 7.61087 | 8.88918 | 12.99674 | 7.56327 | 10.6134 | 18.46915 | +| ResNeXt50_vd_32x4d | 224 | 256 | 7.69065 | 8.94014 | 13.4088 | 7.62044 | 11.03385 | 19.15339 | +| ResNeXt50_64x4d | 224 | 256 | 13.78688 | 15.84655 | 21.79537 | 13.80962 | 18.4712 | 33.49843 | +| ResNeXt50_vd_64x4d | 224 | 256 | 13.79538 | 15.22201 | 22.27045 | 13.94449 | 18.88759 | 34.28889 | +| ResNeXt101_32x4d | 224 | 256 | 16.59777 | 17.93153 | 21.36541 | 16.21503 | 19.96568 | 33.76831 | +| ResNeXt101_vd_32x4d | 224 | 256 | 16.36909 | 17.45681 | 22.10216 | 16.28103 | 20.25611 | 34.37152 | +| ResNeXt101_64x4d | 224 | 256 | 30.12355 | 32.46823 | 38.41901 | 30.4788 | 36.29801 | 68.85559 | +| ResNeXt101_vd_64x4d | 224 | 256 | 30.34022 | 32.27869 | 38.72523 | 30.40456 | 36.77324 | 69.66021 | +| ResNeXt152_32x4d | 224 | 256 | 25.26417 | 26.57001 | 30.67834 | 24.86299 | 29.36764 | 52.09426 | +| ResNeXt152_vd_32x4d | 224 | 256 | 25.11196 | 26.70515 | 31.72636 | 25.03258 | 30.08987 | 52.64429 | +| ResNeXt152_64x4d | 224 | 256 | 46.58293 | 48.34563 | 56.97961 | 46.7564 | 56.34108 | 106.11736 | +| ResNeXt152_vd_64x4d | 224 | 256 | 47.68447 | 48.91406 | 57.29329 | 47.18638 | 57.16257 | 107.26288 | +| SE_ResNet18_vd | 224 | 256 | 1.61823 | 3.1391 | 4.60282 | 1.7691 | 4.19877 | 7.5331 | +| SE_ResNet34_vd | 224 | 256 | 2.67518 | 5.04694 | 7.18946 | 2.88559 | 7.03291 | 12.73502 | +| SE_ResNet50_vd | 224 | 256 | 3.65394 | 7.568 | 12.52793 | 4.28393 | 10.38846 | 18.33154 | +| SE_ResNeXt50_32x4d | 224 | 256 | 9.06957 | 11.37898 | 18.86282 | 8.74121 | 13.563 | 23.01954 | +| SE_ResNeXt50_vd_32x4d | 224 | 256 | 9.25016 | 11.85045 | 25.57004 | 9.17134 | 14.76192 | 19.914 | +| SE_ResNeXt101_32x4d | 224 | 256 | 19.34455 | 20.6104 | 32.20432 | 18.82604 | 25.31814 | 41.97758 | +| SENet154_vd | 224 | 256 | 49.85733 | 54.37267 | 74.70447 | 53.79794 | 66.31684 | 121.59885 | diff --git a/src/PaddleClas/docs/en/models/SwinTransformer_en.md b/src/PaddleClas/docs/en/models/SwinTransformer_en.md new file mode 100644 index 0000000..8f76a66 --- /dev/null +++ b/src/PaddleClas/docs/en/models/SwinTransformer_en.md @@ -0,0 +1,28 @@ +# SwinTransformer +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview +Swin Transformer a new vision Transformer, that capably serves as a general-purpose backbone for computer vision. It is a hierarchical Transformer whose representation is computed with shifted windows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. [Paper](https://arxiv.org/abs/2103.14030)。 + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| SwinTransformer_tiny_patch4_window7_224 | 0.8069 | 0.9534 | 0.812 | 0.955 | 4.5 | 28 | +| SwinTransformer_small_patch4_window7_224 | 0.8275 | 0.9613 | 0.832 | 0.962 | 8.7 | 50 | +| SwinTransformer_base_patch4_window7_224 | 0.8300 | 0.9626 | 0.835 | 0.965 | 15.4 | 88 | +| SwinTransformer_base_patch4_window12_384 | 0.8439 | 0.9693 | 0.845 | 0.970 | 47.1 | 88 | +| SwinTransformer_base_patch4_window7_224[1] | 0.8487 | 0.9746 | 0.852 | 0.975 | 15.4 | 88 | +| SwinTransformer_base_patch4_window12_384[1] | 0.8642 | 0.9807 | 0.864 | 0.980 | 47.1 | 88 | +| SwinTransformer_large_patch4_window7_224[1] | 0.8596 | 0.9783 | 0.863 | 0.979 | 34.5 | 197 | +| SwinTransformer_large_patch4_window12_384[1] | 0.8719 | 0.9823 | 0.873 | 0.982 | 103.9 | 197 | + +[1]: Based on imagenet22k dataset pre-training, and then in imagenet1k dataset transfer learning. + +**Note**: The difference of precision with reference from the difference of data preprocessing. diff --git a/src/PaddleClas/docs/en/models/TNT_en.md b/src/PaddleClas/docs/en/models/TNT_en.md new file mode 100644 index 0000000..fd20736 --- /dev/null +++ b/src/PaddleClas/docs/en/models/TNT_en.md @@ -0,0 +1,19 @@ +# TNT series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +TNT(Transformer-iN-Transformer) series models were proposed by Huawei-Noah in 2021 for modeling both patch-level and pixel-level representation. In each TNT block, an outer transformer block is utilized to process patch embeddings, and an inner transformer block extracts local features from pixel embeddings. The pixel-level feature is projected to the space of patch embedding by a linear transformation layer and then added into the patch. By stacking the TNT blocks, we build the TNT model for image recognition. Experiments on ImageNet benchmark and downstream tasks demonstrate the superiority and efficiency of the proposed TNT architecture. For example, our TNT achieves 81.3% top-1 accuracy on ImageNet which is 1.5% higher than that of DeiT with similar computational cost. [Paper](https://arxiv.org/abs/2103.00112). + + + +## 2. Accuracy, FLOPs and Parameters + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:---------------------:|:----------:|:---------:|:---------:|:---------:| +| TNT_small | 23.8 | 5.2 | 81.12 | 95.56 | \ No newline at end of file diff --git a/src/PaddleClas/docs/en/models/Twins_en.md b/src/PaddleClas/docs/en/models/Twins_en.md new file mode 100644 index 0000000..0096066 --- /dev/null +++ b/src/PaddleClas/docs/en/models/Twins_en.md @@ -0,0 +1,24 @@ +# Twins +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview +The Twins network includes Twins-PCPVT and Twins-SVT, which focuses on the meticulous design of the spatial attention mechanism, resulting in a simple but more effective solution. Since the architecture only involves matrix multiplication, and the current deep learning framework has a high degree of optimization for matrix multiplication, the architecture is very efficient and easy to implement. Moreover, this architecture can achieve excellent performance in a variety of downstream vision tasks such as image classification, target detection, and semantic segmentation. [Paper](https://arxiv.org/abs/2104.13840). + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| pcpvt_small | 0.8082 | 0.9552 | 0.812 | - | 3.7 | 24.1 | +| pcpvt_base | 0.8242 | 0.9619 | 0.827 | - | 6.4 | 43.8 | +| pcpvt_large | 0.8273 | 0.9650 | 0.831 | - | 9.5 | 60.9 | +| alt_gvt_small | 0.8140 | 0.9546 | 0.817 | - | 2.8 | 24 | +| alt_gvt_base | 0.8294 | 0.9621 | 0.832 | - | 8.3 | 56 | +| alt_gvt_large | 0.8331 | 0.9642 | 0.837 | - | 14.8 | 99.2 | + +**Note**:The difference in accuracy from Reference is due to the difference in data preprocessing. diff --git a/src/PaddleClas/docs/en/models/ViT_and_DeiT_en.md b/src/PaddleClas/docs/en/models/ViT_and_DeiT_en.md new file mode 100644 index 0000000..d8e36b9 --- /dev/null +++ b/src/PaddleClas/docs/en/models/ViT_and_DeiT_en.md @@ -0,0 +1,41 @@ +# ViT and DeiT series +--- +## Catalogue + +* [1. Overview](#1) +* [2. Accuracy, FLOPs and Parameters](#2) + + +## 1. Overview + +ViT(Vision Transformer) series models were proposed by Google in 2020. These models only use the standard transformer structure, completely abandon the convolution structure, splits the image into multiple patches and then inputs them into the transformer, showing the potential of transformer in the CV field.。[Paper](https://arxiv.org/abs/2010.11929)。 + +DeiT(Data-efficient Image Transformers) series models were proposed by Facebook at the end of 2020. Aiming at the problem that the ViT models need large-scale dataset training, the DeiT improved them, and finally achieved 83.1% Top1 accuracy on ImageNet. More importantly, using convolution model as teacher model, and performing knowledge distillation on these models, the Top1 accuracy of 85.2% can be achieved on the ImageNet dataset. + + +## 2. Accuracy, FLOPs and Parameters + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ViT_small_patch16_224 | 0.7769 | 0.9342 | 0.7785 | 0.9342 | | | +| ViT_base_patch16_224 | 0.8195 | 0.9617 | 0.8178 | 0.9613 | | | +| ViT_base_patch16_384 | 0.8414 | 0.9717 | 0.8420 | 0.9722 | | | +| ViT_base_patch32_384 | 0.8176 | 0.9613 | 0.8166 | 0.9613 | | | +| ViT_large_patch16_224 | 0.8323 | 0.9650 | 0.8306 | 0.9644 | | | +| ViT_large_patch16_384 | 0.8513 | 0.9736 | 0.8517 | 0.9736 | | | +| ViT_large_patch32_384 | 0.8153 | 0.9608 | 0.815 | - | | | + + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| DeiT_tiny_patch16_224 | 0.718 | 0.910 | 0.722 | 0.911 | | | +| DeiT_small_patch16_224 | 0.796 | 0.949 | 0.799 | 0.950 | | | +| DeiT_base_patch16_224 | 0.817 | 0.957 | 0.818 | 0.956 | | | +| DeiT_base_patch16_384 | 0.830 | 0.962 | 0.829 | 0.972 | | | +| DeiT_tiny_distilled_patch16_224 | 0.741 | 0.918 | 0.745 | 0.919 | | | +| DeiT_small_distilled_patch16_224 | 0.809 | 0.953 | 0.812 | 0.954 | | | +| DeiT_base_distilled_patch16_224 | 0.831 | 0.964 | 0.834 | 0.965 | | | +| DeiT_base_distilled_patch16_384 | 0.851 | 0.973 | 0.852 | 0.972 | | | + + +Params, FLOPs, Inference speed and other information are coming soon. diff --git a/src/PaddleClas/docs/en/models/index.rst b/src/PaddleClas/docs/en/models/index.rst new file mode 100644 index 0000000..4642eb1 --- /dev/null +++ b/src/PaddleClas/docs/en/models/index.rst @@ -0,0 +1,30 @@ +models +================================ + +.. toctree:: + :maxdepth: 2 + + DPN_DenseNet_en.md + models_intro_en.md + RepVGG_en.md + EfficientNet_and_ResNeXt101_wsl_en.md + ViT_and_DeiT_en.md + SwinTransformer_en.md + Others_en.md + SEResNext_and_Res2Net_en.md + ESNet_en.md + HRNet_en.md + ReXNet_en.md + Inception_en.md + TNT_en.md + RedNet_en.md + DLA_en.md + ResNeSt_RegNet_en.md + PP-LCNet_en.md + HarDNet_en.md + ResNet_and_vd_en.md + LeViT_en.md + Mobile_en.md + MixNet_en.md + Twins_en.md + PVTV2_en.md diff --git a/src/PaddleClas/docs/en/models_training/classification_en.md b/src/PaddleClas/docs/en/models_training/classification_en.md new file mode 100644 index 0000000..0a9bce2 --- /dev/null +++ b/src/PaddleClas/docs/en/models_training/classification_en.md @@ -0,0 +1,360 @@ +# Image Classification + +------ + +Image Classification is a fundamental task that classifies the image by semantic information and assigns it to a specific label. Image Classification is the foundation of Computer Vision tasks, such as object detection, image segmentation, object tracking and behavior analysis. Image Classification has comprehensive applications, including face recognition and smart video analysis in the security and protection field, traffic scenario recognition in the traffic field, image retrieval and electronic photo album classification in the internet industry, and image recognition in the medical industry. + +Generally speaking, Image Classification attempts to comprehend an entire image as a whole by feature engineering and assigns labels by a classifier. Hence, how to extract the features of image is the essential part. Before deep learning, the most used classification method is the Bag of Words model. However, Image Classification based on deep learning can learn the hierarchical feature description by supervised and unsupervised learning, replacing the manually image feature selection. Recently, Convolution Neural Network in deep learning has an awesome performance in the image field. CNN uses the pixel information as the input to get the all information of images. Additionally, since the model uses convolution to extract features, and the output is classification result. Thus, this kind of end-to-end method achieves ideal performance and is applied widely. + +Image Classification is a very basic but important field in the subject of computer vision. Its research results have always influenced the development of computer vision and even deep learning. Image classification has many sub-fields, such as multi-label image classification and fine-grained image classification. Here is only a brief description of single-label image classification. + +See [here](../algorithm_introduction/image_classification_en.md) for the detailed introduction of image classification algorithms. + +## Catalogue + +- [1. Dataset Introduction](#1) + - [1.1 ImageNet-1k](#1.1) + - [1.2 CIFAR-10/CIFAR-100](#1.2) +- [2. Image Classification Process](#2) + - [2.1 Data and Its Preprocessing](#2.1) + - [2.2 Prepare the Model](#2.2) + - [2.3 Train the Model](#2.3) + - [2.4 Evaluate the Model](#2.4) +- [3. Application Methods](#3) + - [3.1 Training and Evaluation on CPU or Single GPU](#3.1) + - [3.1.1 Model Training](#3.1.1) + - [3.1.2 Model Finetuning](#3.1.2) + - [3.1.3 Resume Training](#3.1.3) + - [3.1.4 Model Evaluation](#3.1.4) + - [3.2 Training and Evaluation on Linux+ Multi-GPU](#3.2) + - [3.2.1 Model Training](#3.2.1) + - [3.2.2 Model Finetuning](#3.2.2) + - [3.2.3 Resume Training](#3.2.3) + - [3.2.4 Model Evaluation](#3.2.4) + - [3.3 Use the Pre-trained Model to Predict](#3.3) + - [3.4 Use the Inference Model to Predict](#3.4) + + + +## 1. Dataset Introduction + + + +### 1.1 ImageNet-1k + +The ImageNet is a large-scale visual database for the research of visual object recognition. More than 14 million images have been annotated manually to point out objects in the picture in this project, and at least more than 1 million images provide bounding box. ImageNet-1k is a subset of the ImageNet dataset, which contains 1000 categories. The training set contains 1281167 image data, and the validation set contains 50,000 image data. Since 2010, the ImageNet project has held an image classification competition every year, which is the ImageNet Large-scale Visual Recognition Challenge (ILSVRC). The dataset used in the challenge is ImageNet-1k. So far, ImageNet-1k has become one of the most important data sets for the development of computer vision, and it promotes the development of the entire computer vision. The initialization models of many computer vision downstream tasks are based on the weights trained on this dataset. + + + +### 1.2 CIFAR-10/CIFAR-100 + +The CIFAR-10 dataset consists of 60,000 color images in 10 categories, with an image resolution of 32x32, and each category has 6000 images, including 5000 in the training set and 1000 in the validation set. 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships and trucks. The CIFAR-100 data set is an extension of CIFAR-10. It consists of 60,000 color images in 100 classes, with an image resolution of 32x32, and each class has 600 images, including 500 in the training set and 100 in the validation set. Researchers can try different algorithms quickly because these two data sets are small in scale. These two datasets are also commonly used data sets for testing the quality of models in the image classification field. + + + +## 2. Image Classification Process + +The prepared training data is preprocessed and then passed through the image classification model. The output of the model and the real label are used in a cross-entropy loss function. This loss function describes the convergence direction of the model. Then the corresponding gradient descent for the final loss function is calculated and returned to the model, which update the weight of the model by optimizers. Finally, an image classification model can be obtained. + + + +### 2.1 Data Preprocessing + +The quality and quantity of data often determine the performance of a model. In the field of image classification, data includes images and labels. In most cases, labeled data is scarce, so the amount of data is difficult to reach the level of saturation of the model. In order to enable the model to learn more image features, a lot of image transformation or data augmentation is required before the image enters the model, so as to ensure the diversity of input image data and ensure that the model has better generalization capabilities. PaddleClas provides standard image transformation for training ImageNet-1k, and also provides 8 data augmentation methods. For related codes, please refer to [data preprocess](../../../ppcls/data/preprocess)，The configuration file refer to [Data Augmentation Configuration File](../../../ppcls/configs/ImageNet/DataAugment). For related algorithms, please refer to [data augment algorithms](../algorithm_introduction/DataAugmentation_en.md). + + + +### 2.2 Prepare the Model + +After the data is determined, the model often determines the upper limit of the final accuracy. In the field of image classification, classic models emerge in an endless stream. PaddleClas provides 35 series and a total of 164 ImageNet pre-trained models. For specific accuracy, speed and other indicators, please refer to [Backbone Network Introduction](../algorithm_introduction/ImageNet_models_en.md). + + + +### 2.3 Train + +After preparing the data and model, you can start training the model and update the parameters of the model. After many iterations, a trained model can finally be obtained for image classification tasks. The training process of image classification requires a lot of experience and involves the setting of many hyperparameters. PaddleClas provides a series of [training tuning methods](./train_strategy_en.md), which can quickly help you obtain a high-precision model. + +PaddleClas support training with VisualDL to visualize the metric. VisualDL is a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, PR curves , ROC curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently. For more information, please refer to [VisualDL](../others/VisualDL_en.md). + + + +### 2.4 Evaluation + +After a model is trained, the evaluation results of the model on the validation set can determine the performance of the model. The evaluation index is generally Top1-Acc or Top5-Acc. The higher the index, the better the model performance. + + + +## 3. Application Methods + +Please refer to [Installation](../installation/install_paddleclas_en.md) to setup environment at first, and prepare flower102 dataset by following the instruction mentioned in the [Quick Start](../quick_start/quick_start_classification_new_user_en.md). + +So far, PaddleClas supports the following training/evaluation environments: + +``` +└── CPU/Single GPU + ├── Linux + └── Windows + +└── Multi card GPU + └── Linux +``` + + + +### 3.1 Training and Evaluation on CPU or Single GPU + +If training and evaluation are performed on CPU or single GPU, it is recommended to use the `tools/train.py` and `tools/eval.py`. For training and evaluation in multi-GPU environment on Linux, please refer to [3.2 Training and evaluation on Linux+GPU](#3.2). + + + +#### 3.1.1 Model Training + +After preparing the configuration file, The training process can be started in the following way. + +```shell +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Arch.pretrained=False \ + -o Global.device=gpu +``` + +Among them, `-c` is used to specify the path of the configuration file, `-o` is used to specify the parameters needed to be modified or added, `-o Arch.pretrained=False` means to not using pre-trained models. `-o Global.device=gpu` means to use GPU for training. If you want to use the CPU for training, you need to set `Global.device` to `cpu`. + +Of course, you can also directly modify the configuration file to update the configuration. For specific configuration parameters, please refer to [Configuration Document](config_description_en.md). + +The output log examples are as follows: + +- If mixup or cutmix is used in training, top-1 and top-k (default by 5) will not be printed in the log: + + ``` + ... + [Train][Epoch 3/20][Avg]CELoss: 6.46287, loss: 6.46287 + ... + [Eval][Epoch 3][Avg]CELoss: 5.94309, loss: 5.94309, top1: 0.01961, top5: 0.07941 + ... + ``` + +- If mixup or cutmix is not used during training, in addition to the above information, top-1 and top-k (The default is 5) will also be printed in the log: + + ``` + ... + [Train][Epoch 3/20][Avg]CELoss: 6.12570, loss: 6.12570, top1: 0.01765, top5: 0.06961 + ... + [Eval][Epoch 3][Avg]CELoss: 5.40727, loss: 5.40727, top1: 0.07549, top5: 0.20980 + ... + ``` + +During training, you can view loss changes in real time through `VisualDL`, see [VisualDL](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.2/docs/en/extension/VisualDL_en.md) for details. + + + +#### 3.1.2 Model Finetuning + +After correcting config file, you can load pretrained model weight to finetune. The command is as follows: + +```shell +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Arch.pretrained=True \ + -o Global.device=gpu +``` + +Among them,`Arch.pretrained` is used to set the address to load the pretrained weights. When using it, you need to replace it with your own pretrained weights' path, or you can modify the path directly in the configuration file. You can also set it into `True` to use pretrained weights that trained in ImageNet1k. + +We also provide a lot of pre-trained models trained on the ImageNet-1k dataset. For the model list and download address, please refer to the [model library overview](../algorithm_introduction/ImageNet_models_en.md). + + + +#### 3.1.3 Resume Training + +If the training process is terminated for some reasons, you can also load the checkpoints to continue training. + +```shell +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.checkpoints="./output/MobileNetV3_large_x1_0/epoch_5" \ + -o Global.device=gpu +``` + +The configuration file does not need to be modified. You only need to add the `Global.checkpoints` parameter during training, which represents the path of the checkpoints. The parameter weights, learning rate, optimizer and other information will be loaded using this parameter. + +**Note**: + +- The `-o Global.checkpoints` parameter does not need to include the suffix of the checkpoints. The above training command will generate the checkpoints as shown below during the training process. If you want to continue training from the epoch `5`, Just set the `Global.checkpoints` to `../output/MobileNetV3_large_x1_0/epoch_5`, PaddleClas will automatically fill in the `pdopt` and `pdparams` suffixes. Files in the output directory are structured as follows： + + ``` + output + ├── MobileNetV3_large_x1_0 + │ ├── best_model.pdopt + │ ├── best_model.pdparams + │ ├── best_model.pdstates + │ ├── epoch_1.pdopt + │ ├── epoch_1.pdparams + │ ├── epoch_1.pdstates + . + . + . + ``` + + + +#### 3.1.4 Model Evaluation + +The model evaluation process can be started as follows. + +```shell +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.pretrained_model=./output/MobileNetV3_large_x1_0/best_model +``` + +The above command will use `./configs/quick_start/MobileNetV3_large_x1_0.yaml` as the configuration file to evaluate the model `./output/MobileNetV3_large_x1_0/best_model`. You can also set the evaluation by changing the parameters in the configuration file, or you can update the configuration with the `-o` parameter, as shown above. + +Some of the configurable evaluation parameters are described as follows: + +- `Arch.name`：Model name +- `Global.pretrained_model`：The path of the model file to be evaluated + +**Note：** When loading the model to be evaluated, you only need to specify the path of the model file stead of the suffix. PaddleClas will automatically add the `.pdparams` suffix, such as [3.1.3 Resume Training](#3.1.3). + +When loading the model to be evaluated, you only need to specify the path of the model file stead of the suffix. PaddleClas will automatically add the `.pdparams` suffix, such as [3.1.3 Resume Training](https://github.com/PaddlePaddle/PaddleClas/blob/ develop/docs/zh_CN/models_training/classification.md#3.1.3). + + + +### 3.2 Training and Evaluation on Linux+ Multi-GPU + +If you want to run PaddleClas on Linux with GPU, it is highly recommended to use `paddle.distributed.launch` to start the model training script(`tools/train.py`) and evaluation script(`tools/eval.py`), which can start on multi-GPU environment more conveniently. + + + +#### 3.2.1 Model Training + +The training process can be started in the following way. `paddle.distributed.launch` specifies the GPU running card number by setting `gpus`: + +```shell +# PaddleClas initiates multi-card multi-process training via launch + +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml +``` + +The format of output log information is the same as above, see [3.1.1 Model training](#3.1.1) for details. + + + +#### 3.2.2 Model Finetuning + +After configuring the yaml file, you can finetune it by loading the pretrained weights. The command is as below. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Arch.pretrained=True +``` + +Among them, `Arch.pretrained` is set to `True` or `False`. It also can be used to set the address to load the pretrained weights. When using it, you need to replace it with your own pretrained weights' path, or you can modify the path directly in the configuration file. + +There contains a lot of examples of model finetuning in the [new user version](../quick_start/quick_start_classification_new_user_en.md) and [professional version](../quick_start/quick_start_classification_professional_en.md) of PaddleClas Trial in 30 mins. You can refer to this tutorial to finetune the model on a specific dataset. + + + +#### 3.2.3 Resume Training + +If the training process is terminated for some reasons, you can also load the checkpoints to continue training. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.checkpoints="./output/MobileNetV3_large_x1_0/epoch_5" \ + -o Global.device=gpu +``` + +The configuration file does not need to be modified. You only need to add the `Global.checkpoints` parameter during training, which represents the path of the checkpoints. The parameter weights, learning rate, optimizer and other information will be loaded using this parameter as described in [3.1.3 Resume training](#3.1.3). + + + +#### 3.2.4 Model Evaluation + +The model evaluation process can be started as follows. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.pretrained_model=./output/MobileNetV3_large_x1_0/best_model +``` + +About parameter description, see [3.1.4 Model evaluation](#3.1.4) for details. + + + +### 3.3 Use the Pre-trained Model to Predict + +After the training is completed, you can predict by using the pre-trained model obtained by the training. A complete example is provided in `tools/infer/infer.py` of the model library, run the following command to conduct model prediction: + +``` +python3 tools/infer.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Infer.infer_imgs=dataset/flowers102/jpg/image_00001.jpg \ + -o Global.pretrained_model=./output/MobileNetV3_large_x1_0/best_model +``` + +Parameters： + +- `Infer.infer_imgs`：The path of the image file or folder to be predicted. +- `Global.pretrained_model`：Weight file path, such as`./output/MobileNetV3_large_x1_0/best_model` + + + +### 3.4 Use the Inference Model to Predict + +By exporting the inference model，PaddlePaddle supports inference using prediction engines, which will be introduced next. Firstly, you should export inference model using `tools/export_model.py`. + +```shell +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.pretrained_model=output/MobileNetV3_large_x1_0/best_model +``` + +Among them, `Global.pretrained_model` parameter is used to specify the model file path that does not need to include the file suffix name.（such as [3.1.3 Resume Training](#3.1.3)）。 + +The above command will generate the model structure file (`inference.pdmodel`) and the model weight file (`inference.pdiparams`), and then the inference engine can be used for inference: + +Go to the deploy directory: + +``` +cd deploy +``` + +Using inference engine to inference. Because the mapping file of ImageNet1k dataset is `class_id_map_file` by default, here it should be set to None. + +```shell +python3 python/predict_cls.py \ + -c configs/inference_cls.yaml \ + -o Global.infer_imgs=../dataset/flowers102/jpg/image_00001.jpg \ + -o Global.inference_model_dir=../inference/ \ + -o PostProcess.Topk.class_id_map_file=None +``` + +Among them： + +- `Global.infer_imgs`：The path of the image file to be predicted. +- `Global.inference_model_dir`：Model structure file path, such as `../inference/`. +- `Global.use_tensorrt`：Whether to use the TesorRT, default by `False`. +- `Global.use_gpu`：Whether to use the GPU, default by `True`. +- `Global.enable_mkldnn`：Wheter to use `MKL-DNN`, default by `False`. It is valid when `Global.use_gpu` is `False`. +- `Global.use_fp16`：Whether to enable FP16, default by `False`. + +Note: If you want to use `Transformer` series models, such as `DeiT_***_384`, `ViT_***_384`, etc.，please pay attention to the input size of model, and need to set `resize_short=384`, `resize=384`. + +If you want to evaluate the speed of the model, it is recommended to enable TensorRT to accelerate for GPU, and MKL-DNN for CPU. diff --git a/src/PaddleClas/docs/en/models_training/config_description_en.md b/src/PaddleClas/docs/en/models_training/config_description_en.md new file mode 100644 index 0000000..d002547 --- /dev/null +++ b/src/PaddleClas/docs/en/models_training/config_description_en.md @@ -0,0 +1,270 @@ +# Configuration Instruction + +------ + +## Introdction + +The parameters in the PaddleClas configuration file(`ppcls/configs/*.yaml`)are described for you to customize or modify the hyperparameter configuration more quickly. + +## Details + +### Catalogue + +- [1. Classification model](#1) + - [1.1 Global Configuration](#1.1) + - [1.2 Architecture](#1.2) + - [1.3 Loss function](#1.3) + - [1.4 Optimizer](#1.4) + - [1.5 Data reading module(DataLoader)](#1.5) + - [1.5.1 dataset](#1.5.1) + - [1.5.2 sampler](#1.5.2) + - [1.5.3 loader](#1.5.3) + - [1.6 Evaluation metric](#1.6) + - [1.7 Inference](#1.7) +- [2. Distillation model](#2) + - [2.1 Architecture](#2.1) + - [2.2 Loss function](#2.2) + - [2.3 Evaluation metric](#2.3) +- [3. Recognition model](#3) + - [3.1 Architechture](#3.1) + - [3.2 Evaluation metric](#3.2) + + + +### 1. Classification model + +Here the configuration of `ResNet50_vd` on`ImageNet-1k`is used as an example to explain the each parameter in detail. [Configure Path](https://github.com/PaddlePaddle/PaddleClas/blob/develop/ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml). + + +#### 1.1 Global Configuration + +| Parameter name | Specific meaning | Defult value | Optional value | +| ------------------ | ------------------------------------------------------- | ---------------- | ----------------- | +| checkpoints | Breakpoint model path for resuming training | null | str | +| pretrained_model | Pre-trained model path | null | str | +| output_dir | Save model path | "./output/" | str | +| save_interval | How many epochs to save the model at each interval | 1 | int | +| eval_during_train | Whether to evaluate at training | True | bool | +| eval_interval | How many epochs to evaluate at each interval | 1 | int | +| epochs | Total number of epochs in training | | int | +| print_batch_step | How many mini-batches to print out at each interval | 10 | int | +| use_visualdl | Whether to visualize the training process with visualdl | False | bool | +| image_shape | Image size | [3，224，224] | list, shape: (3,) | +| save_inference_dir | Inference model save path | "./inference" | str | +| eval_mode | Model of eval | "classification" | "retrieval" | + +**Note**：The http address of pre-trained model can be filled in the `pretrained_model` + + +#### 1.2 Architecture + +| Parameter name | Specific meaning | Defult value | Optional value | +| -------------- | ----------------- | ------------ | --------------------- | +| name | Model Arch name | ResNet50 | PaddleClas model arch | +| class_num | Category number | 1000 | int | +| pretrained | Pre-trained model | False | bool， str | + +**Note**: Here pretrained can be set to True or False, so does the path of the weights. In addition, the pretrained is disabled when Global.pretrained_model is also set to the corresponding path. + + +#### 1.3 Loss function + +| Parameter name | Specific meaning | Defult value | Optional value | +| -------------- | ------------------------------------------- | ------------ | ---------------------- | +| CELoss | cross-entropy loss function | —— | —— | +| CELoss.weight | The weight of CELoss in the whole Loss | 1.0 | float | +| CELoss.epsilon | The epsilon value of label_smooth in CELoss | 0.1 | float，between 0 and 1 | + + +#### 1.4 Optimizer + +| Parameter name | Specific meaning | Defult value | Optional value | +| ----------------- | -------------------------------- | ------------ | -------------------------------------------------- | +| name | optimizer method name | "Momentum" | Other optimizer including "RmsProp" | +| momentum | momentum value | 0.9 | float | +| lr.name | method of dropping learning rate | "Cosine" | Other dropping methods of "Linear" and "Piecewise" | +| lr.learning_rate | initial value of learning rate | 0.1 | float | +| lr.warmup_epoch | warmup rounds | 0 | int，such as 5 | +| regularizer.name | regularization method name | "L2" | ["L1", "L2"] | +| regularizer.coeff | regularization factor | 0.00007 | float | + +**Note**：The new parameters may be different when `lr.name` is different , as when `lr.name=Piecewise`, the following parameters need to be added: + +``` + lr: + name: Piecewise + learning_rate: 0.1 + decay_epochs: [30, 60, 90] + values: [0.1, 0.01, 0.001, 0.0001] +``` + +Referring to [learning_rate.py](https://github.com/PaddlePaddle/PaddleClas/blob/develop/ppcls/optimizer/learning_rate.py) for adding method and parameters. + + +#### 1.5 Data reading module(DataLoader) + + +##### 1.5.1 dataset + +| Parameter name | Specific meaning | Defult value | Optional value | +| ------------------- | ------------------------------------ | ----------------------------------- | ------------------------------ | +| name | The name of the class to read the data | ImageNetDataset | VeriWild and other Dataet type | +| image_root | The path where the dataset is stored | ./dataset/ILSVRC2012/ | str | +| cls_label_path | data label list | ./dataset/ILSVRC2012/train_list.txt | str | +| transform_ops | data preprocessing for single images | —— | —— | +| batch_transform_ops | Data preprocessing for batch images | —— | —— | + +The parameter meaning of transform_ops: + +| Function name | Parameter name | Specific meaning | +| -------------- | -------------- | --------------------- | +| DecodeImage | to_rgb | data to RGB | +| | channel_first | image data by CHW | +| RandCropImage | size | Random crop | +| RandFlipImage | | Random flip | +| NormalizeImage | scale | Normalize scale value | +| | mean | Normalize mean value | +| | std | normalized variance | +| | order | Normalize order | +| CropImage | size | crop size | +| ResizeImage | resize_short | resize by short edge | + +The parameter meaning of batch_transform_ops: + +| Function name | Parameter name | Specific meaning | +| ------------- | -------------- | --------------------------------------- | +| MixupOperator | alpha | Mixup parameter value，the larger the value, the stronger the augment | + + +##### 1.5.2 sampler + +| Parameter name | Specific meaning | Default value | Optional value | +| -------------- | ------------------------------------------------------------ | ----------------------- | -------------------------------------------------- | +| name | sampler type | DistributedBatchSampler | DistributedRandomIdentitySampler and other Sampler | +| batch_size | batch size | 64 | int | +| drop_last | Whether to drop the last data that does reach the batch-size | False | bool | +| shuffle | whether to shuffle the data | True | bool | + +##### 1.5.3 loader + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| ----------------- | ---------------------------- | --------------- | ---------------- | +| num_workers | Number of data read threads | 4 | int | +| use_shared_memory | Whether to use shared memory | True | bool | + + +#### 1.6 Evaluation metric + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| -------------- | ---------------- | --------------- | ---------------- | +| TopkAcc | TopkAcc | [1, 5] | list, int | + + +#### 1.7 Inference + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| ----------------------------- | --------------------------------- | ------------------------------------- | ---------------- | +| infer_imgs | Image address to be inferred | docs/images/whl/demo.jpg | str | +| batch_size | batch size | 10 | int | +| PostProcess.name | Post-process name | Topk | str | +| PostProcess.topk | topk value | 5 | int | +| PostProcess.class_id_map_file | mapping file of class id and name | ppcls/utils/imagenet1k_label_list.txt | str | + +**Note**：The interpretation of `transforms` in the Infer module refers to the interpretation of`transform_ops`in the dataset in the data reading module. + + +### 2. Distillation model + +**Note**：Here the training configuration of `MobileNetV3_large_x1_0` on `ImageNet-1k` distilled MobileNetV3_small_x1_0 is used as an example to explain the meaning of each parameter in detail. [Configure path](https://github.com/PaddlePaddle/PaddleClas/blob/develop/ppcls/configs/ImageNet/Distillation/mv3_large_x1_0_distill_mv3_small_x1_0.yaml). Only parameters that are distinct from the classification model are introduced here. + + +#### 2.1 Architecture + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| ------------------ | --------------------------------------------------------- | ---------------------- | ---------------------------------- | +| name | model arch name | DistillationModel | —— | +| class_num | category number | 1000 | int | +| freeze_params_list | freeze_params_list | [True, False] | list | +| models | model list | [Teacher, Student] | list | +| Teacher.name | teacher model name | MobileNetV3_large_x1_0 | PaddleClas model | +| Teacher.pretrained | teacher model pre-trained weights | True | Boolean or pre-trained weight path | +| Teacher.use_ssld | whether teacher model pretrained weights are ssld weights | True | Boolean | +| infer_model_name | type of the model being inferred | Student | Teacher | + +**Note**： + +1. list is represented in yaml as follows: + +``` + freeze_params_list: + - True + - False +``` + +2.Student's parameters are similar and will not be repeated. + + +#### 2.2 Loss function + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| ----------------------------------- | ------------------------------------------------------------ | --------------- | ---------------- | +| DistillationCELoss | Distillation's cross-entropy loss function | —— | —— | +| DistillationCELoss.weight | Loss weight | 1.0 | float | +| DistillationCELoss.model_name_pairs | ["Student", "Teacher"] | —— | —— | +| DistillationGTCELoss.weight | Distillation's cross-entropy loss function of model and true Label | —— | —— | +| DistillationGTCELos.weight | Loss weight | 1.0 | float | +| DistillationCELoss.model_names | Model names with real label for cross-entropy | ["Student"] | —— | + + +#### 2.3 Evaluation metric + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| ----------------------------- | ------------------- | ---------------------------- | ---------------- | +| DistillationTopkAcc | DistillationTopkAcc | including model_key and topk | —— | +| DistillationTopkAcc.model_key | the evaluated model | "Student" | "Teacher" | +| DistillationTopkAcc.topk | Topk value | [1, 5] | list, int | + +**Note**： `DistillationTopkAcc` has the same meaning as `TopkAcc`, except that it is only used in distillation tasks. + + +### 3. Recognition model + +**Note**：The training configuration of`ResNet50` on`LogoDet-3k` is used here as an example to explain the meaning of each parameter in detail. [configure path](https://github.com/PaddlePaddle/PaddleClas/blob/develop/ppcls/configs/Logo/ResNet50_ReID.yaml). Only parameters that are distinct from the classification model are presented here. + + +#### 3.1 Architechture + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| ---------------------- | ------------------------------------------------------------ | --------------------------- | ------------------------------------------------------------ | +| name | Model arch | "RecModel" | ["RecModel"] | +| infer_output_key | inference output value | “feature” | ["feature", "logits"] | +| infer_add_softmax | softmaxwhether to add softmax to infercne | False | [True, False] | +| Backbone.name | Backbone name | ResNet50_last_stage_stride1 | other backbone provided by PaddleClas | +| Backbone.pretrained | Backbone pre-trained model | True | Boolean value or pre-trained model path | +| BackboneStopLayer.name | The name of the output layer in Backbone | True | The`full_name`of the feature output layer in Backbone | +| Neck.name | The name of the Neck part | VehicleNeck | the dictionary structure to be passed in, the specific input parameters for the Neck network layer | +| Neck.in_channels | Input dimension size of the Neck part | 2048 | the size is the same as BackboneStopLayer.name | +| Neck.out_channels | Output the dimension size of the Neck part, i.e. feature dimension size | 512 | int | +| Head.name | Network Head part nam | CircleMargin | Arcmargin. Etc | +| Head.embedding_size | Feature dimension size | 512 | Consistent with Neck.out_channels | +| Head.class_num | number of classes | 3000 | int | +| Head.margin | margin value in CircleMargin | 0.35 | float | +| Head.scale | scale value in CircleMargin | 64 | int | + +**Note**： + +1.In PaddleClas, the `Neck` part is the connection part between Backbone and embedding layer, and `Head` part is the connection part between embedding layer and classification layer.。 + +2.`BackboneStopLayer.name` can be obtained by visualizing the model, visualization can be referred to [Netron](https://github.com/lutzroeder/netron) or [visualdl](https://github.com/PaddlePaddle/VisualDL). + +3.Calling tools/export_model.py will convert the model weights to inference model, where the infer_add_softmax parameter will control whether to add the Softmax activation function afterwards, the code default is True (the last output layer in the classification task will be connected to the Softmax activation function). In the recognition task, the activation function is not required for the feature layer, so it should be set to False here. + + + + +#### 3.2 Evaluation metric + +| Parameter name | Specific meaning | Default meaning | Optional meaning | +| -------------- | --------------------------- | --------------- | ---------------- | +| Recallk | Recall rate | [1, 5] | list, int | +| mAP | Average retrieval precision | None | None | diff --git a/src/PaddleClas/docs/en/models_training/index.rst b/src/PaddleClas/docs/en/models_training/index.rst new file mode 100644 index 0000000..1d27e65 --- /dev/null +++ b/src/PaddleClas/docs/en/models_training/index.rst @@ -0,0 +1,10 @@ +models_training +================================ + +.. toctree:: + :maxdepth: 2 + + config_description_en.md + recognition_en.md + classification_en.md + train_strategy_en.md diff --git a/src/PaddleClas/docs/en/models_training/recognition_en.md b/src/PaddleClas/docs/en/models_training/recognition_en.md new file mode 100644 index 0000000..57f2b12 --- /dev/null +++ b/src/PaddleClas/docs/en/models_training/recognition_en.md @@ -0,0 +1,341 @@ +# Image Recognition + + + +Image recognition, in PaddleClas, means that the system is able to recognize the label of a given query image. Broadly speaking, image classification falls under image recognition. But unlike ordinary image recognition, it can only discriminate the learned categories and require retraining to add new ones. The image recognition in PaddleClas, however, only need to update the corresponding search library to identify the category of the unfamiliar images without retraining the model, which not only significantly promotes the usability of the recognition system but also reduces the demand for model updates, facilitating users' deployment of the application. + +For an image to be queried, the image recognition process in PaddleClas is divided into three main parts: + +1. Mainbody Detection: for a given query image, the mainbody detector first identifies the object, thus removing useless background information to improve the recognition accuracy. +2. Feature Extraction: for each candidate region of mainbody detection, feature extraction is performed by the feature model +3. Vector Search: the extracted features are compared with the vectors in the feature gallery for similarity to obtain their label information + +The feature gallery is built in advance using the labeled image datasets. The complete image recognition system is shown in the figure below. + +![img](../../images/structure.jpg) + +To experience the whole image recognition system, or learn how to build a feature gallery, please refer to [Quick Start of Image Recognition](../quick_start/quick_start_recognition_en.md), which explains the overall application process. The following parts expound on the training part of the above three steps. + +Please first refer to the [Installation Guide](../installation/install_paddleclas_en.md) to configure the runtime environment. + +## Catalogue + +- [1. Mainbody Detection](#1) +- [2. Feature Model Training](#2) + - [2.1. Data Preparation](#2.1) + - [2. 2 Single GPU-based Training and Evaluation](#2.2) + - [2.2.1 Model Training](#2.2.1) + - [2.2.2 Resume Training](#2.2.2) + - [2.2.3 Model Evaluation](#2.2.3) + - [2.3 Export Inference Model](#2.3) +- [3. Vector Search](#3) +- [4. Basic Knowledge](#4) + + + +## 1. Mainbody Detection + +The mainbody detection training is based on [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection/tree/develop), the only difference is that all the detection boxes in the mainbody detection task belong to the foreground, but it is necessary to modify `category_id` of the detection box in the annotation file to 1, while changing the `categories` mapping table in the whole annotation file to the following format, i.e., the whole category mapping table contains only `foreground`. + +``` +[{u'id': 1, u'name': u'foreground', u'supercategory': u'foreground'}] +``` + +For more information about the training method of mainbody detection, please refer to: [PaddleDetection Training Tutorial](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md#4-训练). + +For more information on the introduction and download of the model provided in PaddleClas for body detection, please refer to: [PaddleDetection Tutorial](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/zh_CN/image_recognition_pipeline/mainbody_detection.md). + + + +## 2. Feature Model Training + + + +### 2.1 Data Preparation + +- Go to PaddleClas directory. + +``` +## linux or mac， $path_to_PaddleClas indicates the root directory of PaddleClas，which the user needs to modify according to their real directory +cd $path_to_PaddleClas +``` + +- Go to the `dataset`. which the user needs to modify according to their real directory [CUB_200_2011](http://vision.ucsd.edu/sites/default/files/WelinderEtal10_CUB-200.pdf), which is a fine grid dataset with 200 different types of birds. Firstly, we need to download the dataset. For download, please refer to [Official Website](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html). + +```shell +# linux or mac +cd dataset + +# Copy the downloaded data into a directory. +cp {Data storage path}/CUB_200_2011.tgz . + +# Unzip +tar -xzvf CUB_200_2011.tgz + +# Go to CUB_200_2011 +cd CUB_200_2011 +``` + +When using the dataset for image retrieval, we usually use the first 100 classes as the training set, and the last 100 classes as the testing set, so we need to process those data so as to adapt the model training of image retrieval. + +```shell +# Create train and test directories +mkdir train && mkdir test + +# Divide data into training set with the first 100 classes and testing set with the last 100 classes. +ls images | awk -F "." '{if(int($1)<101)print "mv images/"$0" train/"int($1)}' | sh +ls images | awk -F "." '{if(int($1)>100)print "mv images/"$0" test/"int($1)}' | sh + +# Generate train_list and test_list +tree -r -i -f train | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > train_list.txt +tree -r -i -f test | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > test_list.txt +``` + +So far, we have the training set (in the `train` catalog), testing set (in the `test` catalog), `train_list.txt` and `test_list.txt` of `CUB_200_2011`. + +After data preparation, the `train` directory of `CUB_200_2011` should be： + +``` +├── 1 +│ ├── Black_Footed_Albatross_0001_796111.jpg +│ ├── Black_Footed_Albatross_0002_55.jpg + ... +├── 10 +│ ├── Red_Winged_Blackbird_0001_3695.jpg +│ ├── Red_Winged_Blackbird_0005_5636.jpg +... +``` + +`train_list.txt` should be： + +``` +train/99/Ovenbird_0137_92639.jpg 99 1 +train/99/Ovenbird_0136_92859.jpg 99 2 +train/99/Ovenbird_0135_93168.jpg 99 3 +train/99/Ovenbird_0131_92559.jpg 99 4 +train/99/Ovenbird_0130_92452.jpg 99 5 +... +``` + +The separators are shown as spaces, and the meaning of those three columns of data are the directory, label and unique id of training sets. + +The format of testing set is the same as the one of training set. + +**Note**： + +- When the gallery dataset and query dataset are the same, in order to remove the first data retrieved (the retrieved images themselves do not need to be evaluated), each data needs to correspond to a unique id for subsequent evaluation of metrics such as mAP, recall@1, etc. Please refer to [Introduction to image retrieval datasets](#Introduction to Image Retrieval Datasets) for the analysis of gallery datasets and query datasets, and [Image retrieval evaluation metrics](#Image Retrieval Evaluation Metrics) for the evaluation of mAP, recall@1, etc. + +Back to `PaddleClas` root directory. + +```shell +# linux or mac +cd ../../ +``` + + + +### 2.2 Single GPU-based Training and Evaluation + +For training and evaluation on a single GPU, the `tools/train.py` and `tools/eval.py` scripts are recommended. + +PaddleClas support training with VisualDL to visualize the metric. VisualDL is a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, PR curves , ROC curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently. For more information, please refer to [VisualDL](../others/VisualDL_en.md). + + + +#### 2.2.1 Model Training + +Once you have prepared the configuration file, you can start training the image retrieval task in the following way. the method used by PaddleClas to train the image retrieval is metric learning, referring to [metric learning](#metric learning) for more explanations. + +```shell +# Single GPU +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Arch.Backbone.pretrained=True \ + -o Global.device=gpu +# Multi GPU +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Arch.Backbone.pretrained=True \ + -o Global.device=gpu +``` + +`-c` is used to specify the path to the configuration file, and `-o` is used to specify the parameters that need to be modified or added, where `-o Arch.Backbone.pretrained=True` indicates that the Backbone part uses the pre-trained model. In addtion,`Arch.Backbone.pretrained` can also specify the address of a specific model weight file, which needs to be replaced with the path to your own pre-trained model weight file when using it. `-o Global.device=gpu` indicates that the GPU is used for training. If you want to use a CPU for training, you need to set `Global.device` to `cpu`. + +For more detailed training configuration, you can also modify the corresponding configuration file of the model directly. Refer to the [configuration document](config_description_en.md) for specific configuration parameters. + +Run the above commands to check the output log, an example is as follows: + +```` +``` +... +[Train][Epoch 1/50][Avg]CELoss: 6.59110, TripletLossV2: 0.54044, loss: 7.13154 +... +[Eval][Epoch 1][Avg]recall1: 0.46962, recall5: 0.75608, mAP: 0.21238 +... +``` +```` + +The Backbone here is MobileNetV1, if you want to use other backbone, you can rewrite the parameter `Arch.Backbone.name`, for example by adding `-o Arch.Backbone.name={other Backbone}` to the command. In addition, as the input dimension of the `Neck` section differs between models, replacing a Backbone may require rewriting the input size here in a similar way to replacing the Backbone's name. + +In the Training Loss section, [CELoss](../../../ppcls/loss/celoss.py) and [TripletLossV2](../../../ppcls/loss/triplet.py) are used here with the following configuration files: + +``` +Loss: + Train: + - CELoss: + weight: 1.0 + - TripletLossV2: + weight: 1.0 + margin: 0.5 +``` + +The final total Loss is a weighted sum of all Losses, where weight defines the weight of a particular Loss in the final total. If you want to replace other Losses, you can also change the Loss field in the configuration file, for the currently supported Losses please refer to [Loss](../../../ppcls/loss). + + + +#### 2.2.2 Resume Training + +If the training task is terminated for some reasons, it can be recovered by loading the checkpoints weights file and continue training: + +```shell +# Single card +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.checkpoints="./output/RecModel/epoch_5" \ + -o Global.device=gpu +# Multi card +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.checkpoints="./output/RecModel/epoch_5" \ + -o Global.device=gpu +``` + +There is no need to modify the configuration file, just set the `Global.checkpoints` parameter when continuing training, indicating the path to the loaded breakpoint weights file, using this parameter will load both the saved checkpoints weights and information about the learning rate, optimizer, etc. + +**Note**： + +- The `-o Global.checkpoints` parameter need not contain the suffix name of the checkpoint weights file, the above training command will generate the breakpoint weights file as shown below during training, if you want to continue training from breakpoint `5` then the `Global.checkpoints` parameter just needs to be set to `". /output/RecModel/epoch_5"` and PaddleClas will automatically supplement the suffix name. + + ``` + output/ + └── RecModel + ├── best_model.pdopt + ├── best_model.pdparams + ├── best_model.pdstates + ├── epoch_1.pdopt + ├── epoch_1.pdparams + ├── epoch_1.pdstates + . + . + . + ``` + + + +#### 2.2.3 Model Evaluation + +Model evaluation can be carried out with the following commands. + +```shell +# Single card +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.pretrained_model=./output/RecModel/best_model +# Multi card +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.pretrained_model=./output/RecModel/best_model +``` + +The above command will use `./configs/quick_start/MobileNetV1_retrieval.yaml` as a configuration file to evaluate the model obtained from the above training `./output/RecModel/best_model` for evaluation. You can also set up the evaluation by changing the parameters in the configuration file, or you can update the configuration with the `-o` parameter, as shown above. + +Some of the configurable evaluation parameters are introduced as follows. + +- `Arch.name`：the name of the model +- `Global.pretrained_model`：path to the pre-trained model file of the model to be evaluated, unlike `Global.Backbone.pretrained`, the pre-trained model is the weight of the whole model instead of the Backbone only. When it is time to do model evaluation, the weights of the whole model need to be loaded. +- `Metric.Eval`：the metric to be evaluated, by default evaluates recall@1, recall@5, mAP. when you are not going to evaluate a metric, you can remove the corresponding trial marker from the configuration file; when you want to add a certain evaluation metric, you can also refer to [Metric](../../../ppcls/metric/metrics.py) section to add the relevant metric to the configuration file `Metric.Eval`. + +**Note：** + +- When loading the model to be evaluated, the path to the model file needs to be specified, but it is not necessary to include the file suffix, PaddleClas will automatically complete the `.pdparams` suffix, e.g. [2.2.2 Resume Training](#2.2.2). +- Metric learning are generally not evaluated for TopkAcc. + + + +### 2.3 Export Inference Model + +By exporting the inference model, PaddlePaddle supports the transformation of the trained model using prediction with inference engine. + +```shell +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.pretrained_model=output/RecModel/best_model \ + -o Global.save_inference_dir=./inference +``` + +`Global.pretrained_model` is used to specify the model file path, which still does not need to contain the model file suffix (e.g.[2.2.2 Model Recovery Training](#2.2.2)). When executed, it will generate the `./inference` directory, which contains the `inference.pdiparams`,`inference.pdiparams.info`, and`inference.pdmodel` files.`Global.save_inference_dir` allows you to specify the path to export the inference model. The inference model saved here is truncated at the embedding feature level, i.e. the final output of the model is n-dimensional embedding features. + +The above command will generate the model structure file (`inference.pdmodel`) and the model weights file (`inference.pdiparams`), which can then be used for inference using the inference engine. The process of inference using the inference model can be found in [Predictive inference based on the Python prediction engine](../inference_deployment/python_deploy_en.md). + + + +## 3. Vector Search + +Vector search in PaddleClas currently supports the following environments: + +``` +└── CPU + ├── Linux + ├── MacOS + └── Windows +``` + +[Faiss](https://github.com/facebookresearch/faiss) is adopted as a search library, which is an efficient one for feature search and clustering. A variety of similarity search algorithms are integrated in this library to meet different scenarios. In PaddleClas, three search algorithms are supported. + +- **HNSW32**: A graph indexing method boasts high retrieval accuracy and fast speed. However, the feature library only supports the function of adding images, not deleting image features. (Default method) +- **IVF**: An inverted index search method with fast speed but slightly lower precision. The feature library supports functions of adding and deleting image features. +- **FLAT**: A violent search algorithm presenting the highest precision, but slower retrieval speed in face of large data volume. The feature library supports functions of adding and deleting image features. + +See its detailed introduction in the [official document](https://github.com/facebookresearch/faiss/wiki). + +`Faiss` can be installed as follows: + +``` +pip install faiss-cpu==1.7.1post2 +``` + +If the above cannot be properly referenced, please `uninstall` and then `install` again, especially when you are using`windows`. + + + +## 4. Basic Knowledge + +Image retrieval refers to a query image given a specific instance (e.g. a specific target, scene, item, etc.) that contains the same instance from a database image. Unlike image classification, image retrieval solves an open set problem where the training set may not contain the class of the image being recognised. The overall process of image retrieval is: firstly, the images are represented in a suitable feature vector, secondly, a nearest neighbour search is performed on these image feature vectors using Euclidean or Cosine distances to find similar images in the base, and finally, some post-processing techniques can be used to fine-tune the retrieval results and determine information such as the category of the image being recognised. Therefore, the key to determining the performance of an image retrieval algorithm lies in the goodness of the feature vectors corresponding to the images. + + + +- Metric Learning + +Metric learning studies how to learn a distance function on a particular task so that the distance function can help nearest-neighbour based algorithms (kNN, k-means, etc.) to achieve better performance. Deep Metric Learning is a method of metric learning that aims to learn a mapping from the original features to a low-dimensional dense vector space (embedding space) such that similar objects on the embedding space are closer together using commonly used distance functions (Euclidean distance, cosine distance, etc.) ) on the embedding space, while the distances between objects of different classes are not close to each other. Deep metric learning has achieved very successful applications in the field of computer vision, such as face recognition, commodity recognition, image retrieval, pedestrian re-identification, etc. See [HERE](../algorithm_introduction/metric_learning_en.md) for detailed information. + + + +- Introduction to Image Retrieval Datasets + - Training Dataset: used to train the model so that it can learn the image features of the collection. + - Gallery Dataset: used to provide the gallery data for the image retrieval task. The gallery dataset can be the same as the training set or the test set, or different. + - Test Set (Query Dataset): used to test the goodness of the model, usually each test image in the test set is extracted with features, and then matched with the features of the underlying data to obtain recognition results, and then the metrics of the whole test set are calculated based on the recognition results. + + + +- Image Retrieval Evaluation Metrics + + - recall: indicates the number of predicted positive cases with positive labels / the number of cases with positive labels + - recall@1: Number of predicted positive cases in top-1 with positive label / Number of cases with positive label + - recall@5: Number of all predicted positive cases in top-5 retrieved with positive label / Number of cases with positive label + + - mean Average Precision(mAP) + - AP: AP refers to the average precision on different recall rates + - mAP: Average of the APs for all images in the test set diff --git a/src/PaddleClas/docs/en/models_training/train_strategy_en.md b/src/PaddleClas/docs/en/models_training/train_strategy_en.md new file mode 100644 index 0000000..9c5c722 --- /dev/null +++ b/src/PaddleClas/docs/en/models_training/train_strategy_en.md @@ -0,0 +1,122 @@ +# Tricks for Training + +## Catalogue + +- [1. Choice of Optimizers](#1) +- [2. Choice of Learning Rate and Learning Rate Declining Strategy](#2) + - [2.1 Concept of Learning Rate](#2.1) + - [2.2 Learning Rate Decline Strategy](#2.2) + - [2.3 Warmup Strategy](#2.3) +- [3. Choice of Batch_size](#3) +- [4. Choice of Weight_decay](#4) +- [5. Choice of Label_smoothing](#5) +- [6. Change the Crop Area and Stretch Transformation Degree of the Images for Small Models](#6) +- [7. Use Data Augmentation to Improve Accuracy](#7) +- [8. Determine the Tuning Strategy by Train_acc and Test_acc](#8) +- [9. Improve the Accuracy of Your Own Data Set with Existing Pre-trained Models](#9) + + +## 1. Choice of Optimizers +Since the development of deep learning, there have been many researchers working on the optimizer. The purpose of the optimizer is to make the loss function as small as possible, so as to find suitable parameters to complete a certain task. At present, the main optimizers used in model training are SGD, RMSProp, Adam, AdaDelt and so on. The SGD optimizers with momentum is widely used in academia and industry, so most of models we release are trained by SGD optimizer with momentum. But the SGD optimizer with momentum has two disadvantages, one is that the convergence speed is slow, the other is that the initial learning rate is difficult to set, however, if the initial learning rate is set properly and the models are trained in sufficient iterations, the models trained by SGD with momentum can reach higher accuracy compared with the models trained by other optimizers. Some other optimizers with adaptive learning rate such as Adam, RMSProp and so on tent to converge faster, but the final convergence accuracy will be slightly worse. If you want to train a model in faster convergence speed, we recommend you use the optimizers with adaptive learning rate, but if you want to train a model with higher accuracy, we recommend you to use SGD optimizer with momentum. + + +## 2. Choice of Learning Rate and Learning Rate Declining Strategy +The choice of learning rate is related to the optimizer, data set and tasks. Here we mainly introduce the learning rate of training ImageNet-1K with momentum + SGD as the optimizer and the choice of learning rate decline. + + +### 2.1 Concept of Learning Rate +the learning rate is the hyperparameter to control the learning speed, the lower the learning rate, the slower the change of the loss value, though using a low learning rate can ensure that you will not miss any local minimum, but it also means that the convergence speed is slow, especially when the gradient is trapped in a gradient plateau area. + + +### 2.2 Learning Rate Decline Strategy +During training, if we always use the same learning rate, we cannot get the model with highest accuracy, so the learning rate should be adjust during training. In the early stage of training, the weights are in a random initialization state and the gradients are tended to descent, so we can set a relatively large learning rate for faster convergence. In the late stage of training, the weights are close to the optimal values, the optimal value cannot be reached by a relatively large learning rate, so a relatively smaller learning rate should be used. During training, many researchers use the piecewise_decay learning rate reduction strategy, which is a stepwise decline learning rate. For example, in the training of ResNet50, the initial learning rate we set is 0.1, and the learning rate drops to 1/10 every 30 epoches, the total epoches for training is 120. Besides the piecewise_decay, many researchers also proposed other ways to decrease the learning rate, such as polynomial_decay, exponential_decay and cosine_decay and so on, among them, cosine_decay has become the preferred learning rate reduction method for improving model accuracy beacause there is no need to adjust hyperparameters and the robustness is relatively high. The learning rate curves of cosine_decay and piecewise_decay are shown in the following figures, it is easy to observe that during the entire training process, cosine_decay keeps a relatively large learning rate, so its convergence is slower, but the final convergence accuracy is better than the one using piecewise_decay. + +![](../../images/models/lr_decay.jpeg) + +In addition, we can also see from the figures that the number of epoches with a small learning rate in cosine_decay is fewer, which will affect the final accuracy, so in order to make cosine_decay play a better effect, it is recommended to use cosine_decay in large epoched, such as 200 epoches. + + +### 2.3 Warmup Strategy +If a large batch_size is adopted to train nerual network, we recommend you to adopt warmup strategy. as the name suggests, the warmup strategy is to let model learning first warm up, we do not directly use the initial learning rate at the begining of training, instead, we use a gradually increasing learning rate to train the model, when the increasing learning rate reaches the initial learning rate, the learning rate reduction method mentioned in the learning rate reduction strategy is then used to decay the learning rate. Experiments show that when the batch size is large, warmup strategy can improve the accuracy. Some model training with large batch_size such as MobileNetV3 training, we set the epoch in warmup to 5 by default, that is, first in 5 epoches, the learning rate increases from 0 to initial learning rate, then learning rate decay begins. + + +## 3. Choice of Batch_size +Batch_size is an important hyperparameter in training neural networks, batch_size determines how much data is sent to the neural network to for training at a time. In the paper [1], the author found in experiments that when batch_size is linearly related to the learning rate, the convergence accuracy is hardly affected. When training ImageNet data, an initial learning rate of 0.1 are commonly chosen for training, and batch_size is 256, so according to the actual model size and memory, you can set the learning rate to 0.1\*k, batch_size to 256\*k. + + +## 4. Choice of Weight_decay +Overfitting is a common term in machine learning. A simple understanding is that the model performs well on the training data, but it performs poorly on the test data. In the convolutional neural network, there also exists the problem of overfitting. To avoid overfitting, many regular ways have been proposed. Among them, weight_decay is one of the widely used ways to avoid overfitting. After the final loss function, L2 regularization(weight_decay) is added to the loss function, with the help of L2 regularization, the weight of the network tend to choose a smaller value, and finally the parameters in the entire network tends to 0, and the generalization performance of the model is improved accordingly. In different kinds of Deep learning frame, the meaning of L2_decay is the coefficient of L2 regularization, on paddle, the name of this value is L2_decay, so in the following the value is called L2_decay. the larger the coefficient, the more the model tends to be underfitting. In the task of training ImageNet, this parameter is set to 1e-4 in most network. In some small networks such as MobileNet networks, in order to avoid network underfitting, the value is set to 1e-5 ~ 4e-5. Of course, the setting of this value is also related to the specific data set, When the data set is large, the network itself tends to be under-fitted, and the value can be appropriately reduced. When the data set is small, the network tends to overfit itself, so the value can be increased appropriately. The following table shows the accuracy of MobileNetV1_x0_25 using different l2_decay on ImageNet-1k. Since MobileNetV1_x0_25 is a relatively small network, the large l2_decay will make the network tend to be underfitting, so in this network, 3e-5 are better choices compared with 1e-4. + +| Model | L2_decay | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV1_x0_25 | 1e-4 | 43.79%/67.61% | 50.41%/74.70% | +| MobileNetV1_x0_25 | 3e-5 | 47.38%/70.83% | 51.45%/75.45% | + +In addition, the setting of L2_decay is also related to whether other regularization is used during training. If the data argument during the training is more complicated, which means that the training becomes more difficult, L2_decay can be appropriately reduced. The following table shows that the precision of ResNet50 using a different l2_decay on ImageNet-1K. It is easy to observe that after the training becomes difficult, using a smaller l2_decay helps to improve the accuracy of the model. + +| Model | L2_decay | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| ResNet50 | 1e-4 | 75.13%/90.42% | 77.65%/93.79% | +| ResNet50 | 7e-5 | 75.56%/90.55% | 78.04%/93.74% | + +In summary, l2_decay can be adjusted according to specific tasks and models. Usually simple tasks or larger models are recommended to use Larger l2_decay, complex tasks or smaller models are recommended to use smaller l2_decay. + + +## 5. Choice of Label_smoothing +Label_smoothing is a regularization method in deep learning. Its full name is Label Smoothing Regularization (LSR), that is, label smoothing regularization. In the traditional classification task, when calculating the loss function, the real one hot label and the output of the neural network are calculated in cross-entropy formula, the label smoothing aims to make the real one hot label become smooth label, which makes the neural network no longer learn from the hard labels, but the soft labels with a probability value, where the probability of the position corresponding to the category is the largest and the probability of other positions are very small value, specific calculation method can be seen in the paper[2]. In label-smoothing, there is an epsilon parameter describing the degree of softening the label. The larger epsilon, the smaller the probability and smoother the label, on the contrary, the label tends to be hard label. during training on ImageNet-1K, the parameter is usually set to 0.1. In the experiments of training ResNet50, when using label_smoothing, the accuracy is higher than the one without label_smoothing, the following table shows the performance of ResNet50_vd with label smoothing and without label smoothing. + +| Model | Use_label_smoothing | Test acc1 | +|:--:|:--:|:--:| +| ResNet50_vd | 0 | 77.9% | +| ResNet50_vd | 1 | 78.4% | + +But, because label smoothing can be regarded as a regular way, on relatively small models, the accuracy improvement is not obvious or even decreases, the following table shows the accuracy performance of ResNet18 with label smoothing and without label smoothing on ImageNet-1K, it can be clearly seen that after using label smoothing, the accuracy of ResNet has decreased. + +| Model | Use_label_smoohing | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| ResNet18 | 0 | 69.81%/87.70% | 70.98%/89.92% | +| ResNet18 | 1 | 68.00%/86.56% | 70.81%/89.89% | + + +In summary, the use of label_smoohing for larger models can effectively improve the accuracy of the model, and the use of label_smoohing for smaller models may reduce the accuracy of the model, so before deciding whether to use label_smoohing, you need to evaluate the size of the model and the difficulty of the task. + + +## 6. Change the Crop Area and Stretch Transformation Degree of the Images for Small Models +In the standard preprocessing of ImageNet-1k data, two values of scale and ratio are defined in the random_crop function. These two values respectively determine the size of the image crop and the degree of stretching of the image. The default value of scale is 0.08-1(lower_scale-upper_scale), the default value range of ratio is 3/4-4/3(lower_ratio-upper_ratio). In small network training, such data argument will make the network underfitting, resulting in a decrease in accuracy. In order to improve the accuracy of the network, you can make the data argument weaker, that is, increase the crop area of the images or weaken the degree of stretching and transformation of the images, we can achieve weaker image transformation by increasing the value of lower_scale or narrowing the gap between lower_ratio and upper_scale. The following table lists the accuracy of training MobileNetV2_x0_25 with different lower_scale. It can be seen that the training accuracy and validation accuracy are improved after increasing the crop area of the images + +| Model | Scale Range | Train_acc1/acc5 | Test_acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV2_x0_25 | [0.08,1] | 50.36%/72.98% | 52.35%/75.65% | +| MobileNetV2_x0_25 | [0.2,1] | 54.39%/77.08% | 53.18%/76.14% | + + +## 7. Use Data Augmentation to Improve Accuracy +In general, the size of the data set is critical to the performances, but the annotation of images are often more expensive, so the number of annotated images are often scarce. In this case, the data argument is particularly important. In the standard data augmentation for training on ImageNet-1k, two data augmentation methods which are random_crop and random_flip are mainly used. However, in recent years, more and more data augmentation methods have been proposed, such as cutout, mixup, cutmix, AutoAugment, etc. Experiments show that these data augmentation methods can effectively improve the accuracy of the model. The following table lists the performance of ResNet50 in 8 different data augmentation methods. It can be seen that compared to the baseline, all data augmentation methods can be useful for the accuracy of ResNet50, among them cutmix is currently the most effective data argument. More data argument can be seen here[**Data Argument**](https://paddleclas.readthedocs.io/zh_CN/latest/advanced_tutorials/image_augmentation/ImageAugment.html). + +| Model | Data Argument | Test top-1 | +|:--:|:--:|:--:| +| ResNet50 | Baseline | 77.31% | +| ResNet50 | Auto-Augment | 77.95% | +| ResNet50 | Mixup | 78.28% | +| ResNet50 | Cutmix | 78.39% | +| ResNet50 | Cutout | 78.01% | +| ResNet50 | Gridmask | 77.85% | +| ResNet50 | Random-Augment | 77.70% | +| ResNet50 | Random-Erasing | 77.91% | +| ResNet50 | Hide-and-Seek | 77.43% | + + +## 8. Determine the Tuning Strategy by Train_acc and Test_acc +In the process of training the network, the training set accuracy rate and validation set accuracy rate of each epoch are usually printed. Generally speaking, the accuracy of the training set is slightly higher than the accuracy of the validation set or the same are good state in training, but if you find that the accuracy of training set is much higher than the one of validation set, it means that overfitting happens in your task, which need more regularization, such as increase the value of L2_decay, using more data argument or label smoothing and so on. If you find that the accuracy of training set is lower than the one of validation set, it means that underfitting happens in your task, which recommend you to decrease the value of L2_decay, using fewer data argument, increase the area of the crop area of the images, weaken the stretching transformation of the images, remove label_smoothing, etc. + + +## 9. Improve the Accuracy of Your Own Data Set with Existing Pre-trained Models +In the field of computer vision, it has become common to load pre-trained models to train one's own tasks. Compared with starting training from random initialization, loading pre-trained models can often improve the accuracy of specific tasks. In general, the pre-trained model widely used in the industry is obtained from the ImageNet-1k dataset. The fc layer weight of the pre-trained model is a matrix of k\*1000, where k is The number of neurons before, and the weights of the fc layer is not need to load because of the different tasks. In terms of learning rate, if your training data set is particularly small (such as less than 1,000), we recommend that you use a smaller initial learning rate, such as 0.001 (batch_size: 256, the same below), to avoid a large learning rate undermine pre-training weights, if your training data set is relatively large (greater than 100,000), we recommend that you try a larger initial learning rate, such as 0.01 or greater. + + +> If you think this guide is helpful to you, welcome to star our repo:[https://github.com/PaddlePaddle/PaddleClas](https://github.com/PaddlePaddle/PaddleClas) + +## Reference +[1]P. Goyal, P. Dolla ́r, R. B. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. Accurate, large minibatch SGD: training imagenet in 1 hour. CoRR, abs/1706.02677, 2017. + +[2]C.Szegedy,V.Vanhoucke,S.Ioffe,J.Shlens,andZ.Wojna. Rethinking the inception architecture for computer vision. CoRR, abs/1512.00567, 2015. diff --git a/src/PaddleClas/docs/en/others/VisualDL_en.md b/src/PaddleClas/docs/en/others/VisualDL_en.md new file mode 100644 index 0000000..34ff4b8 --- /dev/null +++ b/src/PaddleClas/docs/en/others/VisualDL_en.md @@ -0,0 +1,57 @@ +# Use VisualDL to visualize the training + +--- + +## Catalogue + +* [1. Preface](#1) +* [2. Use VisualDL in PaddleClas](#2) + * [2.1 Set config and start training](#2.1) + * [2.2 Start VisualDL](#2.2) + + +## 1. Preface +VisualDL, a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, PR curves , ROC curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently. For more information, please refer to [VisualDL](https://github.com/PaddlePaddle/VisualDL/). + + +## 2. Use VisualDL in PaddleClas +Now PaddleClas support use VisualDL to visualize the changes of learning rate, loss, accuracy in training. + + +### 2.1 Set config and start training +You only need to set the field `Global.use_visualdl` to `True` in train config: + +```yaml +# config.yaml +Global: +... + use_visualdl: True +... +``` + +PaddleClas will save the VisualDL logs to subdirectory `vdl/` under the output directory specified by `Global.output_dir`. And then you just need to start training normally: + +```shell +python3 tools/train.py -c config.yaml +``` + + +### 2.2 Start VisualDL +After starting the training program, you can start the VisualDL service in a new terminal session: + +```shell + visualdl --logdir ./output/vdl/ +``` + +In the above command, `--logdir` specify the directory of the VisualDL logs produced in training. VisualDL will traverse and iterate to find the subdirectories of the specified directory to visualize all the experimental results. You can also use the following parameters to set the IP and port number of the VisualDL service: + +* `--host`：ip, default is 127.0.0.1 +* `--port`：port, default is 8040 + +More information about the command，please refer to [VisualDL](https://github.com/PaddlePaddle/VisualDL/blob/develop/README.md#2-launch-panel). + +Then you can enter the address `127.0.0.1:8840` and view the training process in the browser: + + +![](../../images/VisualDL/train_loss.png) + diff --git a/src/PaddleClas/docs/en/others/competition_support_en.md b/src/PaddleClas/docs/en/others/competition_support_en.md new file mode 100644 index 0000000..da63d96 --- /dev/null +++ b/src/PaddleClas/docs/en/others/competition_support_en.md @@ -0,0 +1,21 @@ +### Competition Support + +PaddleClas stems from the Baidu's visual business applications and the exploration of frontier visual capabilities. It has helped us achieve leading results in many key events, and continues to promote more frontier visual solutions and landing applications. + + +* 1st place in 2018 Kaggle Open Images V4 object detection challenge + + +* 2nd place in 2019 Kaggle Open Images V5 object detection challenge + * The report is avaiable here: [https://arxiv.org/pdf/1911.07171.pdf](https://arxiv.org/pdf/1911.07171.pdf) + * The pretrained model and code is avaiable here: [source code](https://github.com/PaddlePaddle/PaddleDetection/blob/master/docs/featured_model/OIDV5_BASELINE_MODEL.md) + +* 2nd place in Kacggle Landmark Retrieval Challenge 2019 + * The report is avaiable here: [https://arxiv.org/abs/1906.03990](https://arxiv.org/abs/1906.03990) + * The pretrained model and code is avaiable here: [source code](https://github.com/PaddlePaddle/Research/tree/master/CV/landmark) + +* 2nd place in Kaggle Landmark Recognition Challenge 2019 + * The report is avaiable here: [https://arxiv.org/abs/1906.03990](https://arxiv.org/abs/1906.03990) + * The pretrained model and code is avaiable here: [source code](https://github.com/PaddlePaddle/Research/tree/master/CV/landmark) + +* A-level certificate of three tasks: printed text OCR, face recognition and landmark recognition in the first multimedia information recognition technology competition diff --git a/src/PaddleClas/docs/en/others/feature_visiualization_en.md b/src/PaddleClas/docs/en/others/feature_visiualization_en.md new file mode 100644 index 0000000..1d20525 --- /dev/null +++ b/src/PaddleClas/docs/en/others/feature_visiualization_en.md @@ -0,0 +1,107 @@ +# Guide to Feature Graph Visualization + +------ + +## Catalogue + +- [1. Overview](#1) +- [2. Prepare Work](#2) +- [3. Model Modification](#3) +- [4. Results](#4) + + + + + +## 1. Overview + +The feature graph is the feature representation of the input image in the convolutional network, and the study of which can be beneficial to our understanding and design of the model. Therefore, we employ this tool to visualize the feature graph based on the dynamic graph. + + + +## 2. Prepare Work + +The first step is to select the model to be studied, here we choose ResNet50. Copy the model networking code [resnet.py](../../../ppcls/arch/backbone/legendary_models/resnet.py) to [directory](../../../ppcls/utils/feature_maps_visualization/) and download the [ResNet50 pre-training model](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNet50_pretrained.pdparams) or follow the command below. + +```bash +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNet50_pretrained.pdparams +``` + +For other pre-training models and codes of network structure, please download [model library](../../../ppcls/arch/backbone/) and [pre-training models](../models/models_intro_en.md). + + + +## 3. Model Modification + +Having found the location of the needed feature graph, set self.fm to fetch it out. Here we adopt the feature graph after the stem layer in resnet50 as an example. + +Specify the feature graph to be visualized in the forward function of ResNet50 + +```python + def forward(self, x): + with paddle.static.amp.fp16_guard(): + if self.data_format == "NHWC": + x = paddle.transpose(x, [0, 2, 3, 1]) + x.stop_gradient = True + x = self.stem(x) + fm = x + x = self.max_pool(x) + x = self.blocks(x) + x = self.avg_pool(x) + x = self.flatten(x) + x = self.fc(x) + return x, fm +``` + +Then modify the code [fm_vis.py](../../../ppcls/utils/feature_maps_visualization/fm_vis.py) to import `ResNet50`，instantiating the `net` object: + +``` +from resnet import ResNet50 +net = ResNet50() +``` + +Finally, execute the function + +``` +python tools/feature_maps_visualization/fm_vis.py \ + -i the image you want to test \ + -c channel_num -p pretrained model \ + --show whether to show \ + --interpolation interpolation method\ + --save_path where to save \ + --use_gpu whether to use gpu +``` + +Parameters： + +- `-i`: the path of the image file to be predicted, such as`./test.jpeg` +- `-c`: the dimension of feature graph, such as `5` +- `-p`: path of the weight file, such as `./ResNet50_pretrained` +- `--interpolation`: image interpolation method, default value 1 +- `--save_path`: save path, such as `./tools/` +- `--use_gpu`: whether to enable GPU inference, default value: True + + + +## 4. Results + +- Import the Image： + +![](../../images/feature_maps/feature_visualization_input.jpg) + +- Run the following script of feature graph visualization + +``` +python tools/feature_maps_visualization/fm_vis.py \ + -i ./docs/images/feature_maps/feature_visualization_input.jpg \ + -c 5 \ + -p pretrained/ResNet50_pretrained/ \ + --show=True \ + --interpolation=1 \ + --save_path="./output.png" \ + --use_gpu=False +``` + +- Save the output feature graph as `output.png`, as shown below. + +![](../../images/feature_maps/feature_visualization_output.jpg) diff --git a/src/PaddleClas/docs/en/others/index.rst b/src/PaddleClas/docs/en/others/index.rst new file mode 100644 index 0000000..d106343 --- /dev/null +++ b/src/PaddleClas/docs/en/others/index.rst @@ -0,0 +1,15 @@ +others +================================ + +.. toctree:: + :maxdepth: 2 + + transfer_learning_en.md + train_with_DALI_en.md + VisualDL_en.md + train_on_xpu_en.md + feature_visiualization_en.md + paddle_mobile_inference_en.md + competition_support_en.md + update_history_en.md + versions_en.md diff --git a/src/PaddleClas/docs/en/others/paddle_mobile_inference_en.md b/src/PaddleClas/docs/en/others/paddle_mobile_inference_en.md new file mode 100644 index 0000000..4cb7317 --- /dev/null +++ b/src/PaddleClas/docs/en/others/paddle_mobile_inference_en.md @@ -0,0 +1,130 @@ +# Benchmark on Mobile + +--- + +## Catalogue + +* [1. Introduction](#1) +* [2. Evaluation Steps](#2) + * [2.1 Export the Inference Model](#2.1) + * [2.2 Download Benchmark Binary File](#2.2) + * [2.3 Inference benchmark](#2.3) + * [2.4 Model Optimization and Speed Evaluation](#2.4) + + +## 1. Introduction + +[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) is a set of lightweight inference engine which is fully functional, easy to use and then performs well. Lightweighting is reflected in the use of fewer bits to represent the weight and activation of the neural network, which can greatly reduce the size of the model, solve the problem of limited storage space of the mobile device, and the inference speed is better than other frameworks on the whole. + +In [PaddleClas](https://github.com/PaddlePaddle/PaddleClas), we uses Paddle-Lite to [evaluate the performance on the mobile device](../models/Mobile_en.md), in this section we uses the `MobileNetV1` model trained on the `ImageNet1k` dataset as an example to introduce how to use `Paddle-Lite` to evaluate the model speed on the mobile terminal (evaluated on SD855) + + +## 2. Evaluation Steps + + +### 2.1 Export the Inference Model + +* First you should transform the saved model during training to the special model which can be used to inference, the special model can be exported by `tools/export_model.py`, the specific way of transform is as follows. + +```shell +python tools/export_model.py -m MobileNetV1 -p pretrained/MobileNetV1_pretrained/ -o inference/MobileNetV1 +``` + +Finally the `model` and `parmas` can be saved in `inference/MobileNetV1`. + + +### 2.2 Download Benchmark Binary File + +* Use the adb (Android Debug Bridge) tool to connect the Android phone and the PC, then develop and debug. After installing adb and ensuring that the PC and the phone are successfully connected, use the following command to view the ARM version of the phone and select the pre-compiled library based on ARM version. + +```shell +adb shell getprop ro.product.cpu.abi +``` + +* Download Benchmark_bin File + +```shell +wget -c https://paddle-inference-dist.bj.bcebos.com/PaddleLite/benchmark_0/benchmark_bin_v8 +``` + +If the ARM version is v7, the v7 benchmark_bin file should be downloaded, the command is as follow. + +```shell +wget -c https://paddle-inference-dist.bj.bcebos.com/PaddleLite/benchmark_0/benchmark_bin_v7 +``` + + +### 2.3 Inference benchmark + +After the PC and mobile phone are successfully connected, use the following command to start the model evaluation. + +``` +sh deploy/lite/benchmark/benchmark.sh ./benchmark_bin_v8 ./inference result_armv8.txt true +``` + +Where `./benchmark_bin_v8` is the path of the benchmark binary file, `./inference` is the path of all the models that need to be evaluated, `result_armv8.txt` is the result file, and the final parameter `true` means that the model will be optimized before evaluation. Eventually, the evaluation result file of `result_armv8.txt` will be saved in the current folder. The specific performances are as follows. + +``` +PaddleLite Benchmark +Threads=1 Warmup=10 Repeats=30 +MobileNetV1 min = 30.89100 max = 30.73600 average = 30.79750 + +Threads=2 Warmup=10 Repeats=30 +MobileNetV1 min = 18.26600 max = 18.14000 average = 18.21637 + +Threads=4 Warmup=10 Repeats=30 +MobileNetV1 min = 10.03200 max = 9.94300 average = 9.97627 +``` + +Here is the model inference speed under different number of threads, the unit is FPS, taking model on one threads as an example, the average speed of MobileNetV1 on SD855 is `30.79750FPS`. + + +### 2.4 Model Optimization and Speed Evaluation + +* In II.III section, we mention that the model will be optimized before evaluation, here you can first optimize the model, and then directly load the optimized model for speed evaluation + +* Paddle-Lite +In Paddle-Lite, we provides multiple strategies to automatically optimize the original training model, which contain Quantify, Subgraph fusion, Hybrid scheduling, Kernel optimization and so on. In order to make the optimization more convenient and easy to use, we provide opt tools to automatically complete the optimization steps and output a lightweight, optimal and executable model in Paddle-Lite, which can be downloaded on [Paddle-Lite Model Optimization Page](https://paddle-lite.readthedocs.io/zh/latest/user_guides/model_optimize_tool.html). Here we take `MacOS` as our development environment, download[opt_mac](https://paddlelite-data.bj.bcebos.com/model_optimize_tool/opt_mac) model optimization tools and use the following commands to optimize the model. + + +```shell +model_file="../MobileNetV1/model" +param_file="../MobileNetV1/params" +opt_models_dir="./opt_models" +mkdir ${opt_models_dir} +./opt_mac --model_file=${model_file} \ + --param_file=${param_file} \ + --valid_targets=arm \ + --optimize_out_type=naive_buffer \ + --prefer_int8_kernel=false \ + --optimize_out=${opt_models_dir}/MobileNetV1 +``` + +Where the `model_file` and `param_file` are exported model file and the file address respectively, after transforming successfully, the `MobileNetV1.nb` will be saved in `opt_models` + + + +Use the benchmark_bin file to load the optimized model for evaluation. The commands are as follows. + +```shell +bash benchmark.sh ./benchmark_bin_v8 ./opt_models result_armv8.txt +``` + +Finally the result is saved in `result_armv8.txt` and shown as follow. + +``` +PaddleLite Benchmark +Threads=1 Warmup=10 Repeats=30 +MobileNetV1_lite min = 30.89500 max = 30.78500 average = 30.84173 + +Threads=2 Warmup=10 Repeats=30 +MobileNetV1_lite min = 18.25300 max = 18.11000 average = 18.18017 + +Threads=4 Warmup=10 Repeats=30 +MobileNetV1_lite min = 10.00600 max = 9.90000 average = 9.96177 +``` + + +Taking the model on one threads as an example, the average speed of MobileNetV1 on SD855 is `30.84173FPS`. + +More specific parameter explanation and Paddle-Lite usage can refer to [Paddle-Lite docs](https://paddle-lite.readthedocs.io/zh/latest/)。 diff --git a/src/PaddleClas/docs/en/others/train_on_xpu_en.md b/src/PaddleClas/docs/en/others/train_on_xpu_en.md new file mode 100644 index 0000000..5fb44b1 --- /dev/null +++ b/src/PaddleClas/docs/en/others/train_on_xpu_en.md @@ -0,0 +1,87 @@ +# Introduction to Image Classification Model Kunlun (Continuously updated) + +------ + +## Catalogue + +- [1. Foreword](#1) +- [2. Training of Kunlun](#2) + - [2.1 ResNet50](#2.1) + - [2.2 MobileNetV3](#2.2) + - [2.3 HRNet](#2.3) + - [2.4 VGG16/19](#2.4) + + + +## 1. Forword + +- This document describes the models currently supported by Kunlun and how to train these models on Kunlun devices. To install PaddlePaddle that supports Kunlun, please refer to [install_kunlun](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/09_hardware_support/xpu_docs/paddle_install_cn.html) + + + +## 2. Training of Kunlun + +- See [quick_start](../quick_start/quick_start_classification_new_user_en.md)for data sources and pre-trained models. The training effect of Kunlun is aligned with CPU/GPU. + + + +### 2.1 ResNet50 + +- Command: + +``` +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/ResNet50_vd_finetune_kunlun.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +``` + +The difference with cpu/gpu training lies in the addition of -o use_xpu=True, indicating that the execution is on a Kunlun device. + + + +### 2.2 MobileNetV3 + +- Command： + +``` +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +``` + + + +### 2.3 HRNet + +- Command： + +``` +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/HRNet_W18_C_finetune_kunlun.yaml \ + -o is_distributed=False \ + -o use_xpu=True \ + -o use_gpu=False +``` + + + +### 2.4 VGG16/19 + +- Command： + +``` +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/VGG16_finetune_kunlun.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/VGG19_finetune_kunlun.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +``` diff --git a/src/PaddleClas/docs/en/others/train_with_DALI_en.md b/src/PaddleClas/docs/en/others/train_with_DALI_en.md new file mode 100644 index 0000000..790b469 --- /dev/null +++ b/src/PaddleClas/docs/en/others/train_with_DALI_en.md @@ -0,0 +1,79 @@ +# Train with DALI + +--- + +## Catalogue + +* [1. Preface](#1) +* [2. Installing DALI](#2) +* [3. Using DALI](#3) +* [4. Train with FP16](#4) + + + +## 1. Preface + +[The NVIDIA Data Loading Library](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html) (DALI) is a library for data loading and pre-processing to accelerate deep learning applications. It can build Dataloader of PaddlePaddle. + +Since the Deep Learning relies on a large amount of data in the training stage, these data need to be loaded and preprocessed. These operations are usually executed on the CPU, which limits the further improvement of the training speed, especially when the batch_size is large, which become the bottleneck of training speed. DALI can use GPU to accelerate these operations, thereby further improve the training speed. + + + +## 2. Installing DALI + +DALI only support Linux x64 and version of CUDA is 10.2 or later. + +* For CUDA 10: + + pip install --extra-index-url https://developer.download.nvidia.com/compute/redist nvidia-dali-cuda100 + +* For CUDA 11.0: + + pip install --extra-index-url https://developer.download.nvidia.com/compute/redist nvidia-dali-cuda110 + +For more information about installing DALI, please refer to [DALI](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html). + + + +## 3. Using DALI + +Paddleclas supports training with DALI. Since DALI only supports GPU training, `CUDA_VISIBLE_DEVICES` needs to be set, and DALI needs to occupy GPU memory, so it needs to reserve GPU memory for Dali. To train with DALI, just set the fields in the training config `use_dali = True`, or start the training by the following command: + +```shell +# set the GPUs that can be seen +export CUDA_VISIBLE_DEVICES="0" + +python ppcls/train.py -c ppcls/configs/ImageNet/ResNet/ResNet50.yaml -o Global.use_dali=True +``` + +And you can train with muti-GPUs: + +```shell +# set the GPUs that can be seen +export CUDA_VISIBLE_DEVICES="0,1,2,3" + +# set the GPU memory used for neural network training, generally 0.8 or 0.7, and the remaining GPU memory is reserved for DALI +export FLAGS_fraction_of_gpu_memory_to_use=0.80 + +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + ppcls/train.py \ + -c ./ppcls/configs/ImageNet/ResNet/ResNet50.yaml \ + -o Global.use_dali=True +``` + + + +## 4. Train with FP16 + +On the basis of the above, using FP16 half-precision can further improve the training speed, you can refer to the following command. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +export FLAGS_fraction_of_gpu_memory_to_use=0.8 + +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + ppcls/train.py \ + -c ./ppcls/configs/ImageNet/ResNet/ResNet50_fp16_dygraph.yaml +``` diff --git a/src/PaddleClas/docs/en/others/transfer_learning_en.md b/src/PaddleClas/docs/en/others/transfer_learning_en.md new file mode 100644 index 0000000..05ee6cb --- /dev/null +++ b/src/PaddleClas/docs/en/others/transfer_learning_en.md @@ -0,0 +1,103 @@ +# Transfer learning in image classification + +Transfer learning is an important part of machine learning, which is widely used in various fields such as text and images. Here we mainly introduce transfer learning in the field of image classification, which is often called domain transfer, such as migration of the ImageNet classification model to the specified image classification task, such as flower classification. + +--- + +## Catalogue + +* [1. Hyperparameter search](#1) + * [1.1 Grid search](#1.1) + * [1.2 Bayesian search](#1.2) +* [2. Large-scale image classification](#2) +* [3. Reference](#3) + + +## 1. Hyperparameter search + +ImageNet is the widely used dataset for image classification. A series of empirical hyperparameters have been summarized. High accuracy can be got using the hyperparameters. However, when applied in the specified dataset, the hyperparameters may not be optimal. There are two commonly used hyperparameter search methods that can be used to help us obtain better model hyperparameters. + + +### 1.1 Grid search + +For grid search, which is also called exhaustive search, the optimal value is determined by finding the best solution from all solutions in the search space. The method is simple and effective, but when the search space is large, it takes huge computing resource. + + +### 1.2 Bayesian search + +Bayesian search, which is also called Bayesian optimization, is realized by randomly selecting a group of hyperparameters in the search space. Gaussian process is used to update the hyperparameters, compute their expected mean and variance according to the performance of the previous hyperparameters. The larger the expected mean, the greater the probability of being close to the optimal solution. The larger the expected variance, the greater the uncertainty. Usually, the hyperparameter point with large expected mean is called `exporitation`, and the hyperparameter point with large variance is called `exploration`. Acquisition function is defined to balance the expected mean and variance. The currently selected hyperparameter point is viewed as the optimal position with maximum probability. + +According to the above two search schemes, we carry out some experiments based on fixed scheme and two search schemes on 8 open source datasets. As the experimental scheme in [1], we search for 4 hyperparameters, the search space and The experimental results are as follows: + +a fixed set of parameter experiments and two search schemes on 8 open source data sets. With reference to the experimental scheme of [1], we search for 4 hyperparameters, the search space and the experimental results are as follows: + + +- Fixed scheme. + +``` +lr=0.003，l2 decay=1e-4，label smoothing=False，mixup=False +``` + +- Search space of the hyperparameters. + +``` +lr: [0.1, 0.03, 0.01, 0.003, 0.001, 0.0003, 0.0001] + +l2 decay: [1e-3, 3e-4, 1e-4, 3e-5, 1e-5, 3e-6, 1e-6] + +label smoothing: [False, True] + +mixup: [False, True] +``` + +It takes 196 times for grid search, and takes 10 times less for Bayesian search. The baseline is trained by using ImageNet1k pretrained model based on ResNet50_vd and fixed scheme. The follow shows the experiments. + + +| Dataset | Fix scheme | Grid search | Grid search time | Bayesian search | Bayesian search time| +| ------------------ | -------- | -------- | -------- | -------- | ---------- | +| Oxford-IIIT-Pets | 93.64% | 94.55% | 196 | 94.04% | 20 | +| Oxford-102-Flowers | 96.08% | 97.69% | 196 | 97.49% | 20 | +| Food101 | 87.07% | 87.52% | 196 | 87.33% | 23 | +| SUN397 | 63.27% | 64.84% | 196 | 64.55% | 20 | +| Caltech101 | 91.71% | 92.54% | 196 | 92.16% | 14 | +| DTD | 76.87% | 77.53% | 196 | 77.47% | 13 | +| Stanford Cars | 85.14% | 92.72% | 196 | 92.72% | 25 | +| FGVC Aircraft | 80.32% | 88.45% | 196 | 88.36% | 20 | + + +- The above experiments verify that Bayesian search only reduces the accuracy by 0% to 0.4% under the condition of reducing the number of searches by about 10 times compared to grid search. +- The search space can be expaned easily using Bayesian search. + + +## Large-scale image classification + +In practical applications, due to the lack of training data, the classification model trained on the ImageNet1k data set is often used as the pretrained model for other image classification tasks. In order to further help solve practical problems, based on ResNet50_vd, Baidu open sourced a self-developed large-scale classification pretrained model, in which the training data contains 100,000 categories and 43 million pictures. The pretrained model can be downloaded as follows：[**download link**](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNet50_vd_10w_pretrained.pdparams) + +We conducted transfer learning experiments on 6 self-collected datasets, + +using a set of fixed parameters and a grid search method, in which the number of training rounds was set to 20epochs, the ResNet50_vd model was selected, and the ImageNet pre-training accuracy was 79.12%. The comparison results of the experimental data set parameters and model accuracy are as follows: + + +Fixed scheme： + +``` +lr=0.001，l2 decay=1e-4，label smoothing=False，mixup=False +``` + +| Dataset | Statstics | **Pretrained moel on ImageNet
Top-1(fixed)/Top-1(search)** | **Pretrained moel on large-scale dataset
Top-1(fixed)/Top-1(search)** | +| --------------- | ----------------------------------------- | -------------------------------------------------------- | --------------------------------------------------------- | +| Flowers | class:102
train:5789
valid:2396 | 0.7779/0.9883 | 0.9892/0.9954 | +| Hand-painted stick figures | Class:18
train:1007
valid:432 | 0.8795/0.9196 | 0.9107/0.9219 | +| Leaves | class:6
train:5256
valid:2278 | 0.8212/0.8482 | 0.8385/0.8659 | +| Container vehicle | Class:115
train:4879
valid:2094 | 0.6230/0.9556 | 0.9524/0.9702 | +| Chair | class:5
train:169
valid:78 | 0.8557/0.9688 | 0.9077/0.9792 | +| Geology | class:4
train:671
valid:296 | 0.5719/0.8094 | 0.6781/0.8219 | + +- The above experiments verified that for fixed parameters, compared with the pretrained model on ImageNet, using the large-scale classification model as a pretrained model can help us improve the model performance on a new dataset in most cases. Parameter search can be further helpful to the model performance. + + +## Reference + +[1] Kornblith, Simon, Jonathon Shlens, and Quoc V. Le. "Do better imagenet models transfer better?." *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2019. + +[2] Kolesnikov, Alexander, et al. "Large Scale Learning of General Visual Representations for Transfer." *arXiv preprint arXiv:1912.11370* (2019). diff --git a/src/PaddleClas/docs/en/others/update_history_en.md b/src/PaddleClas/docs/en/others/update_history_en.md new file mode 100644 index 0000000..23c425f --- /dev/null +++ b/src/PaddleClas/docs/en/others/update_history_en.md @@ -0,0 +1,55 @@ +# Release Notes + +- 2021.04.15 + - Add `MixNet` and `ReXNet` pretrained models, `MixNet_L`'s Top-1 Acc on ImageNet-1k reaches 78.6% and `ReXNet_3_0` reaches 82.09%. +- 2021.01.27 + * Add ViT and DeiT pretrained models, ViT's Top-1 Acc on ImageNet reaches 81.05%, and DeiT reaches 85.5%. +- 2021.01.08 + * Add support for whl package and its usage, Model inference can be done by simply install paddleclas using pip. +- 2020.12.16 + * Add support for TensorRT when using cpp inference to obain more obvious acceleration. +- 2020.12.06 + * Add `SE_HRNet_W64_C_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 84.75%. +- 2020.11.23 + * Add `GhostNet_x1_3_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 79.38%. +- 2020.11.09 + * Add `InceptionV3` architecture and pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 79.1%. + +* 2020.10.20 + * Add `Res2Net50_vd_26w_4s_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 83.1%. + * Add `Res2Net101_vd_26w_4s_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 83.9%. + +- 2020.10.12 + * Add Paddle-Lite demo. + +- 2020.10.10 + * Add cpp inference demo. + * Improve FAQ tutorials. + +* 2020.09.17 + * Add `HRNet_W48_C_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 83.62%. + * Add `ResNet34_vd_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 79.72%. + +* 2020.09.07 + * Add `HRNet_W18_C_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 81.16%. + * Add `MobileNetV3_small_x0_35_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 55.55%. + +* 2020.07.14 + * Add `Res2Net200_vd_26w_4s_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 85.13%. + * Add `Fix_ResNet50_vd_ssld_v2` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 84.00%. + +* 2020.06.17 + * Add English documents. + +* 2020.06.12 + * Add support for training and evaluation on Windows or CPU. + +* 2020.05.17 + * Add support for mixed precision training. + +* 2020.05.09 + * Add user guide about Paddle Serving and Paddle-Lite. + * Add benchmark about FP16/FP32 on T4 GPU. + +* 2020.04.14 + * First commit. diff --git a/src/PaddleClas/docs/en/others/versions_en.md b/src/PaddleClas/docs/en/others/versions_en.md new file mode 100644 index 0000000..e983a92 --- /dev/null +++ b/src/PaddleClas/docs/en/others/versions_en.md @@ -0,0 +1,60 @@ +# Version Updates + +------ + +## Catalogue + +- [1. v2.3](#1) +- [2. v2.2](#2) + + + +## 1. v2.3 + +- Model Update + - Add pre-training weights for lightweight models, including detection models and feature models + - Release PP-LCNet series of models, which are self-developed ones designed to run on CPU + - Enable SwinTransformer, Twins, and Deit to support direct training from scrach to achieve thesis accuracy. +- Basic framework capabilities + - Add DeepHash module, which supports feature model to directly export binary features + - Add PKSampler, which tackles the problem that feature models cannot be trained by multiple machines and cards + - Support PaddleSlim: support quantization, pruning training, and offline quantization of classification models and feature models + - Enable legendary models to support intermediate model output + - Support multi-label classification training +- Inference Deployment + - Replace the original feature retrieval library with Faiss to improve platform adaptability + - Support PaddleServing: support the deployment of classification models and image recognition process +- Versions of the Recommendation Library + - python: 3.7 + - PaddlePaddle: 2.1.3 + - PaddleSlim: 2.2.0 + - PaddleServing: 0.6.1 + + + +## 2. v2.2 + +- Model Updates + - Add models including LeViT, Twins, TNT, DLA, HardNet, RedNet, and SwinTransfomer +- Basic framework capabilities + - Divide the classification models into two categories + - legendary models: introduce TheseusLayer base class, add the interface to modify the network function, and support the networking data truncation and output + - model zoo: other common classification models + - Add the support of Metric Learning algorithm + - Add a variety of related loss algorithms, and the basic network module gears (allow the combination with backbone and loss) for convenient use + - Support both the general classification and metric learning-related training + - Support static graph training + - Classification training with dali acceleration supported + - Support fp16 training +- Application Updates + - Add specific application cases and related models of product recognition, vehicle recognition (vehicle fine-grained classification, vehicle ReID), logo recognition, animation character recognition + - Add a complete pipeline for image recognition, including detection module, feature extraction module, and vector search module +- Inference Deployment + - Add Mobius, Baidu's self-developed vector search module, to support the inference deployment of the image recognition system + - Image recognition, build feature library that allows batch_size>1 +- Documents Update + - Add image recognition related documents + - Fix bugs in previous documents +- Versions of the Recommendation Library + - python: 3.7 + - PaddlePaddle: 2.1.2 diff --git a/src/PaddleClas/docs/en/quick_start/index.rst b/src/PaddleClas/docs/en/quick_start/index.rst new file mode 100644 index 0000000..2edd78d --- /dev/null +++ b/src/PaddleClas/docs/en/quick_start/index.rst @@ -0,0 +1,10 @@ +quick_start +================================ + +.. toctree:: + :maxdepth: 2 + + quick_start_classification_new_user_en.md + quick_start_classification_professional_en.md + quick_start_recognition_en.md + quick_start_multilabel_classification_en.md diff --git a/src/PaddleClas/docs/en/quick_start/quick_start_classification_new_user_en.md b/src/PaddleClas/docs/en/quick_start/quick_start_classification_new_user_en.md new file mode 100644 index 0000000..12a5e74 --- /dev/null +++ b/src/PaddleClas/docs/en/quick_start/quick_start_classification_new_user_en.md @@ -0,0 +1,194 @@ +# Trial in 30mins(new users) + +This tutorial is mainly for new users, that is, users who are in the introductory stage of deep learning-related theoretical knowledge, know some python grammar, and can read simple codes. This content mainly includes the use of PaddleClas for image classification network training and model prediction. + +--- + +## Catalogue + +- [1. Basic knowledge](#1) +- [2. Environmental installation and configuration](#2) +- [3. Data preparation and processing](#3) +- [4. Model training](#4) + - [4.1 Use CPU for model training](#4.1) + - [4.1.1 Training without using pre-trained models](#4.1.1) + - [4.1.2 Use pre-trained models for training](#4.1.2) + - [4.2 Use GPU for model training](#4.2) + - [4.2.1 Training without using pre-trained models](#4.2.1) + - [4.2.2 Use pre-trained models for training](#4.2.2) +- [5. Model prediction](#5) + + +## 1. Basic knowledge + +Image classification is a pattern classification problem, which is the most basic task in computer vision. Its goal is to classify different images into different categories. We will briefly explain some concepts that need to be understood during model training. We hope to be helpful to you who are experiencing PaddleClas for the first time: + +- train/val/test dataset represents training set, validation set and test set respectively: + - Training dataset: used to train the model so that the model can recognize different types of features; + - Validation set (val dataset): the test set during the training process, which is convenient for checking the status of the model during the training process; + - Test dataset: After training the model, the test dataset is used to evaluate the results of the model. + +- Pre-trained model + + Using a pre-trained model trained on a larger dataset, that is, the weights of the parameters are preset, can help the model converge faster on the new dataset. Especially for some tasks with scarce training data, when the neural network parameters are very large, we may not be able to fully train the model with a small amount of training data. The method of loading the pre-trained model can be thought of as allowing the model to learn based on a better initial weight, so as to achieve better performance. + +- epoch + + The total number of training epochs of the model. The model passes through all the samples in the training set once, which is an epoch. When the difference between the error rate of the validation set and the error rate of the training set is small, the current number of epochs can be considered appropriate; when the error rate of the validation set first decreases and then becomes larger, it means that the number of epochs is too large and the number of epochs needs to be reduced. Otherwise, the model may overfit the training set. + +- Loss Function + + During the training process, measure the difference between the model output (predicted value) and the ground truth. + +- Accuracy (Acc): indicates the proportion of the number of samples with correct predictions to the total data + + - Top1 Acc: If the classification with the highest probability in the prediction result is correct, it is judged to be correct; + - Top5 Acc: If there is a correct classification among the top 5 probability rankings in the prediction result, it is judged as correct; + + +## 2. Environmental installation and configuration + +For specific installation steps, please refer to [Paddle Installation Document](../installation/install_paddle_en.md), [PaddleClas Installation Document](../installation/install_paddleclas_en.md). + + +## 3. Data preparation and processing + +Enter the PaddleClas directory: + +```shell +# linux or mac， $path_to_PaddleClas represents the root directory of PaddleClas, and users need to modify it according to their real directory. +cd $path_to_PaddleClas +``` + +Enter the `dataset/flowers102` directory, download and unzip the flowers102 dataset: + +```shell +# linux or mac +cd dataset/ +# If you want to download directly from the browser, you can copy the link and visit, then download and unzip +wget https://paddle-imagenet-models-name.bj.bcebos.com/data/flowers102.zip +# unzip +unzip flowers102.zip +``` + +If there is no `wget` command or if you are downloading in the Windows operating system, you need to copy the address to the browser to download, and unzip it to the directory `PaddleClas/dataset/`. + +After the unzip operation is completed, there are three `.txt` files for training and testing under the directory `PaddleClas/dataset/flowers102`: `train_list.txt` (training set, 1020 images), `val_list.txt` (validation Set, 1020 images), `train_extra_list.txt` (larger training set, 7169 images). The format of each line in the file: **image relative path** **image label_id** (note: there is a space between the two columns), and there is also a mapping file for label id and category name: `flowers102_label_list.txt` . + +The image files of the flowers102 dataset are stored in the `dataset/flowers102/jpg` directory. The image examples are as follows: + +
+![](../../images/quick_start/Examples-Flower-102.png) +
+ +Return to the root directory of `PaddleClas`: + +```shell +# linux or mac +cd ../../ +# windoes users can open the PaddleClas root directory +``` + + +## 4. Model training + + +### 4.1 Use CPU for model training + +Since the CPU is used for model training, the calculation speed is slow, so here is ShuffleNetV2_x0_25 as an example. This model has a small amount of calculation and a faster calculation speed on the CPU. But also because the model is small, the accuracy of the trained model will also be limited. + + +#### 4.1.1 Training without using pre-trained models + +```shell +# If you are using the windows operating system, please enter the root directory of PaddleClas in cmd and execute this command: +python tools/train.py -c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml +``` + +- The `-c` parameter is to specify the path of the configuration file for training, and the specific hyperparameters for training can be viewed in the `yaml` file +- The `Global.device` parameter in the `yaml` file is set to `cpu`, that is, the CPU is used for training (if not set, this parameter defaults to `gpu`) +- The `epochs` parameter in the `yaml` file is set to 20, indicating that 20 epoch iterations are performed on the entire data set. It is estimated that the training can be completed in about 20 minutes (different CPUs have slightly different training times). At this time, the training model is not sufficient. To improve the accuracy of the training model, please set this parameter to a larger value, such as **40**, the training time will be extended accordingly + + +#### 4.1.2 Use pre-trained models for training + +```shell +python tools/train.py -c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml -o Arch.pretrained=True +``` + +- The `-o` parameter can be set to `True` or `False`, or it can be the storage path of the pre-training model. When `True` is selected, the pre-training weights will be automatically downloaded to the local. Note: If it is a pre-training model path, do not add: `.pdparams` + +You can compare whether to use the pre-trained model and observe the drop in loss. + + +### 4.2 Use GPU for model training + +Since GPU training is faster and more complex models can be used, take ResNet50_vd as an example. Compared with ShuffleNetV2_x0_25, this model is more computationally intensive, and the accuracy of the trained model will be higher. + +First, you must set the environment variables and use the 0th GPU for training: + +- For Linux users: + + ```shell + export CUDA_VISIBLE_DEVICES=0 + ``` + +- For Windows users + + ```shell + set CUDA_VISIBLE_DEVICES=0 + ``` + + +#### 4.2.1 Training without using pre-trained models + +```shell +python tools/train.py -c ./ppcls/configs/quick_start/ResNet50_vd.yaml +``` + +After the training is completed, the `Top1 Acc` curve of the validation set is shown below, and the highest accuracy rate is 0.2735. + +![](../../images/quick_start/r50_vd_acc.png) + + +#### 4.2.2 Use pre-trained models for training + +Based on ImageNet1k classification pre-trained model for fine-tuning, the training script is as follows: + +```shell +python tools/train.py -c ./ppcls/configs/quick_start/ResNet50_vd.yaml -o Arch.pretrained=True +``` + +**Note**: This training script uses GPU. If you use CPU, you can modify it as shown in [4.1 Use CPU for model training] (#4.1) above. + +The `Top1 Acc` curve of the validation set is shown below. The highest accuracy rate is `0.9402`. After loading the pre-trained model, the accuracy of the flowers102 data set has been greatly improved, and the absolute accuracy has increased by more than 65%. + +![](../../images/quick_start/r50_vd_pretrained_acc.png) + + +## 5. Model prediction + +After the training is completed, the trained model can be used to predict the image category. Take the trained ResNet50_vd model as an example, the prediction code is as follows: + +```shell +cd $path_to_PaddleClas +python tools/infer.py -c ./ppcls/configs/quick_start/ResNet50_vd.yaml -o Infer.infer_imgs=dataset/flowers102/jpg/image_00001.jpg -o Global.pretrained_model=output/ResNet50_vd/best_model +``` + +`-i` indicates the path of a single image. After running successfully, the sample results are as follows: + +`[{'class_ids': [76, 51, 37, 33, 9], 'scores': [0.99998, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00001.jpg', 'label_names': ['passion flower', 'wild pansy', 'great masterwort', 'mexican aster', 'globe thistle']}]` + + +Of course, you can also use the trained ShuffleNetV2_x0_25 model for prediction, the code is as follows: + +```shell +cd $path_to_PaddleClas +python tools/infer.py -c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml -o Infer.infer_imgs=dataset/flowers102/jpg/image_00001.jpg -o Global.pretrained_model=output/ShuffleNetV2_x0_25/best_model +``` + +The `-i` parameter can also be the directory of the image file to be tested (`dataset/flowers102/jpg/`). After running successfully, some sample results are as follows: + +`[{'class_ids': [76, 51, 37, 33, 9], 'scores': [0.99998, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00001.jpg', 'label_names': ['passion flower', 'wild pansy', 'great masterwort', 'mexican aster', 'globe thistle']}, {'class_ids': [76, 51, 37, 33, 32], 'scores': [0.99999, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00002.jpg', 'label_names': ['passion flower', 'wild pansy', 'great masterwort', 'mexican aster', 'love in the mist']}, {'class_ids': [76, 12, 39, 73, 78], 'scores': [0.99998, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00003.jpg', 'label_names': ['passion flower', 'king protea', 'lenten rose', 'rose', 'toad lily']}, {'class_ids': [76, 37, 34, 12, 9], 'scores': [0.86282, 0.11177, 0.00717, 0.00599, 0.00397], 'file_name': 'dataset/flowers102/jpg/image_00004.jpg', 'label_names': ['passion flower', 'great masterwort', 'alpine sea holly', 'king protea', 'globe thistle']}, {'class_ids': [76, 37, 33, 51, 69], 'scores': [0.9999, 1e-05, 1e-05, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00005.jpg', 'label_names': ['passion flower', 'great masterwort', 'mexican aster', 'wild pansy', 'tree poppy']}, {'class_ids': [76, 37, 51, 33, 73], 'scores': [0.99999, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00006.jpg', 'label_names': ['passion flower', 'great masterwort', 'wild pansy', 'mexican aster', 'rose']}, {'class_ids': [76, 37, 12, 91, 30], 'scores': [0.98746, 0.00211, 0.00201, 0.00136, 0.0007], 'file_name': 'dataset/flowers102/jpg/image_00007.jpg', 'label_names': ['passion flower', 'great masterwort', 'king protea', 'bee balm', 'carnation']}, {'class_ids': [76, 37, 81, 77, 72], 'scores': [0.99976, 3e-05, 2e-05, 2e-05, 1e-05], 'file_name': 'dataset/flowers102/jpg/image_00008.jpg', 'label_names': ['passion flower', 'great masterwort', 'clematis', 'lotus', 'water lily']}, {'class_ids': [76, 37, 13, 12, 34], 'scores': [0.99646, 0.00144, 0.00092, 0.00035, 0.00027], 'file_name': 'dataset/flowers102/jpg/image_00009.jpg', 'label_names': ['passion flower', 'great masterwort', 'spear thistle', 'king protea', 'alpine sea holly']}, {'class_ids': [76, 37, 34, 33, 51], 'scores': [0.99999, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00010.jpg', 'label_names': ['passion flower', 'great masterwort', 'alpine sea holly', 'mexican aster', 'wild pansy']}]` + +Among them, the length of the list is the size of batch_size. \ No newline at end of file diff --git a/src/PaddleClas/docs/en/quick_start/quick_start_classification_professional_en.md b/src/PaddleClas/docs/en/quick_start/quick_start_classification_professional_en.md new file mode 100644 index 0000000..fe559fe --- /dev/null +++ b/src/PaddleClas/docs/en/quick_start/quick_start_classification_professional_en.md @@ -0,0 +1,306 @@ +# Trial in 30mins(professional) + +Here is a quick start tutorial for professional users to use PaddleClas on the Linux operating system. The main content is based on the CIFAR-100 data set. You can quickly experience the training of different models, experience loading different pre-trained models, experience the SSLD knowledge distillation solution, and experience data augmentation. Please refer to [Installation Guide](../installation/install_paddleclas_en.md) to configure the operating environment and clone PaddleClas code. + +------ + +## Catalogue + +- [1. Data and model preparation](#1) + - [1.1 Data preparation](#1.1) + - [1.1.1 Prepare CIFAR100](#1.1.1) +- [2. Model training](#2) + - [2.1 Single label training](#2.1) + - [2.1.1 Training without loading the pre-trained model](#2.1.1) + - [2.1.2 Transfer learning](#2.1.2) +- [3. Data Augmentation](#3) + - [3.1 Data augmentation-Mixup](#3.1) +- [4. Knowledge distillation](#4) +- [5. Model evaluation and inference](#5) + - [5.1 Single-label classification model evaluation and inference](#5.1) + - [5.1.1 Single-label classification model evaluation](#5.1.1) + - [5.1.2 Single-label classification model prediction](#5.1.2) + - [5.1.3 Single-label classification uses inference model for model inference](#5.1.3) + + + +## 1. Data and model preparation + + + +### 1.1 Data preparation + + +* Enter the PaddleClas directory. + +``` +cd path_to_PaddleClas +``` + + + +#### 1.1.1 Prepare CIFAR100 + +* Enter the `dataset/` directory, download and unzip the CIFAR100 dataset. + +```shell +cd dataset +wget https://paddle-imagenet-models-name.bj.bcebos.com/data/CIFAR100.tar +tar -xf CIFAR100.tar +cd ../ +``` + + + +## 2. Model training + + + +### 2.1 Single label training + + + +#### 2.1.1 Training without loading the pre-trained model + +* Based on the ResNet50_vd model, the training script is shown below. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" +``` + +The highest accuracy of the validation set is around 0.415. + + + + +#### 2.1.2 Transfer learning + +* Based on ImageNet1k classification pre-training model ResNet50_vd_pretrained (accuracy rate 79.12%) for fine-tuning, the training script is shown below. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" \ + -o Arch.pretrained=True +``` + +The highest accuracy of the validation set is about 0.718. After loading the pre-trained model, the accuracy of the CIFAR100 data set has been greatly improved, with an absolute accuracy increase of 30%. + +* Based on ImageNet1k classification pre-training model ResNet50_vd_ssld_pretrained (accuracy rate of 82.39%) for fine-tuning, the training script is shown below. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" \ + -o Arch.pretrained=True \ + -o Arch.use_ssld=True +``` + +In the final CIFAR100 verification set, the top-1 accuracy is 0.73. Compared with the fine-tuning of the pre-trained model with a top-1 accuracy of 79.12%, the top-1 accuracy of the new data set can be increased by 1.2% again. + +* Replace the backbone with MobileNetV3_large_x1_0 for fine-tuning, the training script is shown below. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV3_large_x1_0_CIFAR100_finetune.yaml \ + -o Global.output_dir="output_CIFAR" \ + -o Arch.pretrained=True +``` + +The highest accuracy of the validation set is about 0.601, which is nearly 12% lower than ResNet50_vd. + + + + +## 3. Data Augmentation + +PaddleClas contains many data augmentation methods, such as Mixup, Cutout, RandomErasing, etc. For specific methods, please refer to [Data augmentation chapter](../algorithm_introduction/DataAugmentation_en.md)。 + + + +### 3.1 Data augmentation-Mixup + +Based on the training method in [Data Augmentation Chapter](../algorithm_introduction/DataAugmentation_en.md) in Section 3.3, combined with Mixup's data augmentation method for training, the specific training script is shown below. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_mixup_CIFAR100_finetune.yaml \ + -o Global.output_dir="output_CIFAR" + +``` + + +The final accuracy on the CIFAR100 verification set is 0.73, and the use of data augmentation can increase the model accuracy by about 1.2% again. + + +* **Note** + +* For other data augmentation configuration files, please refer to the configuration files in `ppcls/configs/ImageNet/DataAugment/`. +* The number of epochs for training CIFAR100 is small, so the accuracy of the validation set may fluctuate by about 1%. + + + + +## 4. Knowledge distillation + + +PaddleClas includes a self-developed SSLD knowledge distillation scheme. For specific content, please refer to [Knowledge Distillation Chapter](../algorithm_introduction/knowledge_distillation_en.md). This section will try to use knowledge distillation technology to train the MobileNetV3_large_x1_0 model. Here we use the ResNet50_vd model trained in section 2.1.2 as the teacher model for distillation. First, save the ResNet50_vd model trained in section 2.1.2 to the specified directory. The script is as follows. + +```shell +mkdir pretrained +cp -r output_CIFAR/ResNet50_vd/best_model.pdparams ./pretrained/ +``` + +The model name, teacher model and student model configuration, pre-training address configuration, and freeze_params configuration in the configuration file are as follows, where the two values in `freeze_params_list` represent whether the teacher model and the student model freeze parameter training respectively. + +```yaml +Arch: + name: "DistillationModel" + # if not null, its lengths should be same as models + pretrained_list: + # if not null, its lengths should be same as models + freeze_params_list: + - True + - False + models: + - Teacher: + name: ResNet50_vd + pretrained: "./pretrained/best_model" + - Student: + name: MobileNetV3_large_x1_0 + pretrained: True +``` + +The loss configuration is as follows, where the training loss is the cross entropy of the output of the student model and the teacher model, and the validation loss is the cross entropy of the output of the student model and the true label. + +```yaml +Loss: + Train: + - DistillationCELoss: + weight: 1.0 + model_name_pairs: + - ["Student", "Teacher"] + Eval: + - DistillationGTCELoss: + weight: 1.0 + model_names: ["Student"] +``` + +The final training script is shown below. + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/R50_vd_distill_MV3_large_x1_0_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" + +``` + + +In the end, the accuracy on the CIFAR100 validation set was 64.4%. Using the teacher model for knowledge distillation, the accuracy of MobileNetV3 increased by 4.3%. + +* **Note** + +* In the distillation process, the pre-trained model used by the teacher model is the training result on the CIFAR100 dataset, and the student model uses the MobileNetV3_large_x1_0 pre-trained model with an accuracy of 75.32% on the ImageNet1k dataset. + * The distillation process does not need to use real labels, so more unlabeled data can be used. In the process of use, you can generate fake `train_list.txt` from unlabeled data, and then merge it with the real `train_list.txt`, You can experience it yourself based on your own data. + + + +## 5. Model evaluation and inference + + + +### 5.1 Single-label classification model evaluation and inference + + + +#### 5.1.1 Single-label classification model evaluation + +After training the model, you can use the following commands to evaluate the accuracy of the model. + +```bash +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.pretrained_model="output_CIFAR/ResNet50_vd/best_model" +``` + + + +#### 5.1.2 Single-label classification model prediction + +After the model training is completed, the pre-trained model obtained by the training can be loaded for model prediction. A complete example is provided in `tools/infer.py`, the model prediction can be completed by executing the following command: + +```python +python3 tools/infer.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Infer.infer_imgs=./dataset/CIFAR100/test/0/0001.png \ + -o Global.pretrained_model=output_CIFAR/ResNet50_vd/best_model +``` + + + +#### 5.1.3 Single-label classification uses inference model for model inference + +We need to export the inference model, PaddlePaddle supports the use of prediction engines for inference. Here, we will introduce how to use the prediction engine for inference: +First, export the trained model to inference model: + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.pretrained_model=output_CIFAR/ResNet50_vd/best_model +``` + +* By default, `inference.pdiparams`, `inference.pdmodel` and `inference.pdiparams.info` files will be generated in the `inference` folder. + +Use prediction engines for inference: + +Enter the deploy directory: + +```bash +cd deploy +``` + +Change the `inference_cls.yaml` file. Since the resolution used for training CIFAR100 is 32x32, the relevant resolution needs to be changed. The image preprocessing in the final configuration file is as follows: + +```yaml +PreProcess: + transform_ops: + - ResizeImage: + resize_short: 36 + - CropImage: + size: 32 + - NormalizeImage: + scale: 0.00392157 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - ToCHWImage: +``` + +Execute the command to make predictions. Since the default `class_id_map_file` is the mapping file of the ImageNet dataset, you need to set None here. + +```bash +python3 python/predict_cls.py \ + -c configs/inference_cls.yaml \ + -o Global.infer_imgs=../dataset/CIFAR100/test/0/0001.png \ + -o PostProcess.Topk.class_id_map_file=None +``` diff --git a/src/PaddleClas/docs/en/quick_start/quick_start_multilabel_classification_en.md b/src/PaddleClas/docs/en/quick_start/quick_start_multilabel_classification_en.md new file mode 100644 index 0000000..39adbb7 --- /dev/null +++ b/src/PaddleClas/docs/en/quick_start/quick_start_multilabel_classification_en.md @@ -0,0 +1,117 @@ +# Quick Start of Multi-label Classification + +Experience the training, evaluation, and prediction of multi-label classification based on the [NUS-WIDE-SCENE](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html) dataset, which is a subset of the NUS-WIDE dataset. Please first install PaddlePaddle and PaddleClas, see [Paddle Installation](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/zh_CN/installation) and [PaddleClas installation](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/zh_CN/installation/install_ paddleclas.md) for more details. + +## Catalogue + +- [1. Data and Model Preparation](#1) +- [2. Model Training](#2) +- [3. Model Evaluation](#3) +- [4. Model Prediction](#4) +- [5. Predictive engine-based Prediction](#5) + - [5.1 Export inference model](#5.1) + - [5.2 Predictive engine-based Prediction](#5.2) + + + +## 1. Data and Model Preparation + +- Go to `PaddleClas`. + +``` +cd path_to_PaddleClas +``` + +- Create and go to `dataset/NUS-WIDE-SCENE`, download and unzip the NUS-WIDE-SCENE dataset. + +``` +mkdir dataset/NUS-WIDE-SCENE +cd dataset/NUS-WIDE-SCENE +wget https://paddle-imagenet-models-name.bj.bcebos.com/data/NUS-SCENE-dataset.tar +tar -xf NUS-SCENE-dataset.tar +``` + +- Return to `PaddleClas` root directory + +``` +cd ../../ +``` + + + +## 2. Model Training + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml +``` + +After training 10 epochs, the best correctness of the validation set should be around 0.95. + + + +## 3. Model Evaluation + +``` +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + + + +## 4. Model Prediction + +``` +python3 tools/infer.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + +Obtain an output silimar to the following: + +``` +[{'class_ids': [6, 13, 17, 23, 26, 30], 'scores': [0.95683, 0.5567, 0.55211, 0.99088, 0.5943, 0.78767], 'file_name': './deploy/images/0517_2715693311.jpg', 'label_names': []}] +``` + + + +## 5. Predictive engine-based Prediction + + + +### 5.1 Export inference model + +``` +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + +The path of the inference model is by default under the current path `. /inference`. + + + +### 5.2 Predictive engine-based Prediction + +Go to the `deploy` first： + +``` +cd ./deploy +``` + +Inference and prediction through predictive engines: + +``` +python3 python/predict_cls.py \ + -c configs/inference_multilabel_cls.yaml +``` + +Obtain an output silimar to the following: + +``` +0517_2715693311.jpg: class id(s): [6, 13, 17, 23, 26, 30], score(s): [0.96, 0.56, 0.55, 0.99, 0.59, 0.79], label_name(s): [] +``` diff --git a/src/PaddleClas/docs/en/quick_start/quick_start_recognition_en.md b/src/PaddleClas/docs/en/quick_start/quick_start_recognition_en.md new file mode 100644 index 0000000..61c6f23 --- /dev/null +++ b/src/PaddleClas/docs/en/quick_start/quick_start_recognition_en.md @@ -0,0 +1,296 @@ +# Quick Start of Recognition + +This tutorial contains 3 parts: Environment Preparation, Image Recognition Experience, and Unknown Category Image Recognition Experience. + +If the image category already exists in the image index database, then you can take a reference to chapter [Image Recognition Experience](#2)，to complete the progress of image recognition；If you wish to recognize unknow category image, which is not included in the index database，you can take a reference to chapter [Unknown Category Image Recognition Experience](#3)，to complete the process of creating an index to recognize it。 + +## Catalogue + +* [1. Enviroment Preparation](#1) +* [2. Image Recognition Experience](#2) + * [2.1 Download and Unzip the Inference Model and Demo Data](#2.1) + * [2.2 Product Recognition and Retrieval](#2.2) + * [2.2.1 Single Image Recognition](#2.2.1) + * [2.2.2 Folder-based Batch Recognition](#2.2.2) +* [3. Unknown Category Image Recognition Experience](#3) + * [3.1 Prepare for the new images and labels](#3.1) + * [3.2 Build a new Index Library](#3.2) + * [3.3 Recognize the Unknown Category Images](#3.3) + + + +## 1. Enviroment Preparation + +* Installation：Please take a reference to [Quick Installation ](../installation/)to configure the PaddleClas environment. + +* Using the following command to enter Folder `deploy`. All content and commands in this section need to be run in folder `deploy`. + + ``` + cd deploy + ``` + + +## 2. Image Recognition Experience + +The detection model with the recognition inference model for the 4 directions (Logo, Cartoon Face, Vehicle, Product), the address for downloading the test data and the address of the corresponding configuration file are as follows. + +| Models Introduction | Recommended Scenarios | inference Model | Predict Config File | Config File to Build Index Database | +| ------------ | ------------- | -------- | ------- | -------- | +| Generic mainbody detection model | General Scenarios |[Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar) | - | - | +| Logo Recognition Model | Logo Scenario | [Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/logo_rec_ResNet50_Logo3K_v1.0_infer.tar) | [inference_logo.yaml](../../../deploy/configs/inference_logo.yaml) | [build_logo.yaml](../../../deploy/configs/build_logo.yaml) | +| Cartoon Face Recognition Model| Cartoon Face Scenario | [Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/cartoon_rec_ResNet50_iCartoon_v1.0_infer.tar) | [inference_cartoon.yaml](../../../deploy/configs/inference_cartoon.yaml) | [build_cartoon.yaml](../../../deploy/configs/build_cartoon.yaml) | +| Vehicle Fine-Grained Classfication Model | Vehicle Scenario | [Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_cls_ResNet50_CompCars_v1.0_infer.tar) | [inference_vehicle.yaml](../../../deploy/configs/inference_vehicle.yaml) | [build_vehicle.yaml](../../../deploy/configs/build_vehicle.yaml) | +| Product Recignition Model | Product Scenario | [Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar) | [inference_product.yaml](../../../deploy/configs/inference_product.yaml) | [build_product.yaml](../../../deploy/configs/build_product.yaml) | +| Vehicle ReID Model | Vehicle ReID Scenario | [Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_reid_ResNet50_VERIWild_v1.0_infer.tar) | - | - | + +| Models Introduction | Recommended Scenarios | inference Model | Predict Config File | Config File to Build Index Database | +| ------------ | ------------- | -------- | ------- | -------- | +| Lightweight generic mainbody detection model | General Scenarios |[Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar) | - | - | +| Lightweight generic recognition model | General Scenarios | [Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar) | [inference_product.yaml](../../../deploy/configs/inference_product.yaml) | [build_product.yaml](../../../deploy/configs/build_product.yaml) | + + +Demo data in this tutorial can be downloaded here: [download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognition_demo_data_en_v1.1.tar). + + +**Attention** +1. If you do not have wget installed on Windows, you can download the model by copying the link into your browser and unzipping it in the appropriate folder; for Linux or macOS users, you can right-click and copy the download link to download it via the `wget` command. +2. If you want to install `wget` on macOS, you can run the following command. +3. The predict config file of the lightweight generic recognition model and the config file to build index database are used for the config of product recognition model of server-side. You can modify the path of the model to complete the index building and prediction. + +```shell +# install homebrew +ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"; +# install wget +brew install wget +``` + +3. If you want to isntall `wget` on Windows, you can refer to [link](https://www.cnblogs.com/jeshy/p/10518062.html). If you want to install `tar` on Windows, you can refer to [link](https://www.cnblogs.com/chooperman/p/14190107.html). + + +* You can download and unzip the data and models by following the command below + +```shell +mkdir models +cd models +# Download and unzip the inference model +wget {Models download link} && tar -xf {Name of the tar archive} +cd .. + +# Download the demo data and unzip +wget {Data download link} && tar -xf {Name of the tar archive} +``` + + + +### 2.1 Download and Unzip the Inference Model and Demo Data + +Take the product recognition as an example, download the detection model, recognition model and product recognition demo data with the following commands. + +```shell +mkdir models +cd models +# Download the generic detection inference model and unzip it +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar && tar -xf ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar +# Download and unpack the inference model +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar && tar -xf product_ResNet50_vd_aliproduct_v1.0_infer.tar +cd .. + +# Download the demo data and unzip it +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognition_demo_data_en_v1.1.tar && tar -xf recognition_demo_data_en_v1.1.tar +``` + +Once unpacked, the `recognition_demo_data_v1.1` folder should have the following file structure. + +``` +├── recognition_demo_data_v1.1 +│ ├── gallery_cartoon +│ ├── gallery_logo +│ ├── gallery_product +│ ├── gallery_vehicle +│ ├── test_cartoon +│ ├── test_logo +│ ├── test_product +│ └── test_vehicle +├── ... +``` + +here, original images to build index are in folder `gallery_xxx`, test images are in folder `test_xxx`. You can also access specific folder for more details. + +The `models` folder should have the following file structure. + +``` +├── product_ResNet50_vd_aliproduct_v1.0_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +├── ppyolov2_r50vd_dcn_mainbody_v1.0_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +``` + +**Attention** +If you want to use the lightweight generic recognition model, you need to re-extract the features of the demo data and re-build the index. The way is as follows: + +```shell +python3.7 python/build_gallery.py -c configs/build_product.yaml -o Global.rec_inference_model_dir=./models/general_PPLCNet_x2_5_lite_v1.0_infer +``` + + +### 2.2 Product Recognition and Retrieval + +Take the product recognition demo as an example to show the recognition and retrieval process (if you wish to try other scenarios of recognition and retrieval, replace the corresponding configuration file after downloading and unzipping the corresponding demo data and model to complete the prediction). + +**Note:** `faiss` is used as search library. The installation method is as follows： + +``` +pip install faiss-cpu==1.7.1post2 +``` + +If error happens when using `import faiss`, please uninstall `faiss` and reinstall it, especially on `Windows`. + + + +#### 2.2.1 Single Image Recognition + +Run the following command to identify and retrieve the image `./recognition_demo_data_v1.1/test_product/daoxiangcunjinzhubing_6.jpg` for recognition and retrieval + +```shell +# use the following command to predict using GPU. +python3.7 python/predict_system.py -c configs/inference_product.yaml +# use the following command to predict using CPU +python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.use_gpu=False +``` + + +The image to be retrieved is shown below. + +![](../../images/recognition/product_demo/query/daoxiangcunjinzhubing_6.jpg) + + +The final output is shown below. + +``` +[{'bbox': [287, 129, 497, 326], 'rec_docs': 'Daoxaingcun Golden Piggie Cake', 'rec_scores': 0.8309420347213745}, {'bbox': [99, 242, 313, 426], 'rec_docs': 'Daoxaingcun Golden Piggie Cake', 'rec_scores': 0.7245651483535767}] +``` + + +where bbox indicates the location of the detected object, rec_docs indicates the labels corresponding to the label in the index dabase that are most similar to the detected object, and rec_scores indicates the corresponding confidence. + + +The detection result is also saved in the folder `output`, for this image, the visualization result is as follows. + +![](../../images/recognition/product_demo/result/daoxiangcunjinzhubing_6_en.jpg) + + + +#### 2.2.2 Folder-based Batch Recognition + +If you want to predict the images in the folder, you can directly modify the `Global.infer_imgs` field in the configuration file, or you can also modify the corresponding configuration through the following `-o` parameter. + +```shell +# using the following command to predict using GPU, you can append `-o Global.use_gpu=False` to predict using CPU. +python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./recognition_demo_data_v1.1/test_product/" +``` + + +The results on the screen are shown as following. + +``` +... +[{'bbox': [37, 29, 123, 89], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.6163763999938965}, {'bbox': [153, 96, 235, 175], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5279821157455444}] +[{'bbox': [735, 562, 1133, 851], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5588355660438538}] +[{'bbox': [124, 50, 230, 129], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.6980369687080383}] +[{'bbox': [0, 0, 275, 183], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5818190574645996}] +[{'bbox': [400, 1179, 905, 1537], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.9814301133155823}, {'bbox': [295, 713, 820, 1046], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.9496176242828369}, {'bbox': [153, 236, 694, 614], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.8395382761955261}] +[{'bbox': [544, 4, 1482, 932], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5143815279006958}] +... +``` + +All the visualization results are also saved in folder `output`. + + +Furthermore, the recognition inference model path can be changed by modifying the `Global.rec_inference_model_dir` field, and the path of the index to the index databass can be changed by modifying the `IndexProcess.index_dir` field. + + + +## 3. Recognize Images of Unknown Category + +To recognize the image `./recognition_demo_data_v1.1/test_product/anmuxi.jpg`, run the command as follows: + +```shell +python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./recognition_demo_data_v1.1/test_product/anmuxi.jpg" +``` + +The image to be retrieved is shown below. + +![](../../images/recognition/product_demo/query/anmuxi.jpg) + +The output is empty. + +Since the index infomation is not included in the corresponding index databse, the recognition result is empty or not proper. At this time, we can complete the image recognition of unknown categories by constructing a new index database. + +When the index database cannot cover the scenes we actually recognise, i.e. when predicting images of unknown categories, we need to add similar images of the corresponding categories to the index databasey, thus completing the recognition of images of unknown categories ，which does not require retraining. + + +### 3.1 Prepare for the new images and labels + +First, you need to copy the images which are similar with the image to retrieval to the original images for the index database. The command is as follows. + +```shell +cp -r ../docs/images/recognition/product_demo/gallery/anmuxi ./recognition_demo_data_/gallery_product/gallery/ +``` + +Then you need to create a new label file which records the image path and label information. Use the following command to create a new file based on the original one. + +```shell +# copy the file +cp recognition_demo_data_v1.1/gallery_product/data_file.txt recognition_demo_data_v1.1/gallery_product/data_file_update.txt +``` + +Then add some new lines into the new label file, which is shown as follows. + +``` +gallery/anmuxi/001.jpg Anmuxi Ambrosial Yogurt +gallery/anmuxi/002.jpg Anmuxi Ambrosial Yogurt +gallery/anmuxi/003.jpg Anmuxi Ambrosial Yogurt +gallery/anmuxi/004.jpg Anmuxi Ambrosial Yogurt +gallery/anmuxi/005.jpg Anmuxi Ambrosial Yogurt +gallery/anmuxi/006.jpg Anmuxi Ambrosial Yogurt +``` + +Each line can be splited into two fields. The first field denotes the relative image path, and the second field denotes its label. The `delimiter` is `tab` here. + + + +### 3.2 Build a new Index Base Library + +Use the following command to build the index to accelerate the retrieval process after recognition. + +```shell +python3.7 python/build_gallery.py -c configs/build_product.yaml -o IndexProcess.data_file="./recognition_demo_data_v1.1/gallery_product/data_file_update.txt" -o IndexProcess.index_dir="./recognition_demo_data_v1.1/gallery_product/index_update" +``` + +Finally, the new index information is stored in the folder`./recognition_demo_data_v1.1/gallery_product/index_update`. Use the new index database for the above index. + + + +### 3.3 Recognize the Unknown Category Images + +To recognize the image `./recognition_demo_data_v1.1/test_product/anmuxi.jpg`, run the command as follows. + +```shell +# using the following command to predict using GPU, you can append `-o Global.use_gpu=False` to predict using CPU. +python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./recognition_demo_data_v1.1/test_product/anmuxi.jpg" -o IndexProcess.index_dir="./recognition_demo_data_v1.1/gallery_product/index_update" +``` + +The output is as follows: + +``` +[{'bbox': [243, 80, 523, 522], 'rec_docs': 'Anmuxi Ambrosial Yogurt', 'rec_scores': 0.5570770502090454}] +``` + +The final recognition result is `Anmuxi Ambrosial Yogurt`, which is corrrect, the visualization result is as follows. + +![](../../images/recognition/product_demo/result/anmuxi_en.jpg) + diff --git a/src/PaddleClas/docs/images/10w_cls.png b/src/PaddleClas/docs/images/10w_cls.png new file mode 100644 index 0000000..9db4833 Binary files /dev/null and b/src/PaddleClas/docs/images/10w_cls.png differ diff --git a/src/PaddleClas/docs/images/PP-LCNet/PP-LCNet-Acc.png b/src/PaddleClas/docs/images/PP-LCNet/PP-LCNet-Acc.png new file mode 100644 index 0000000..57c6da6 Binary files /dev/null and b/src/PaddleClas/docs/images/PP-LCNet/PP-LCNet-Acc.png differ diff --git a/src/PaddleClas/docs/images/PP-LCNet/PP-LCNet.png b/src/PaddleClas/docs/images/PP-LCNet/PP-LCNet.png new file mode 100644 index 0000000..df35aa7 Binary files /dev/null and b/src/PaddleClas/docs/images/PP-LCNet/PP-LCNet.png differ diff --git a/src/PaddleClas/docs/images/VisualDL/train_loss.png b/src/PaddleClas/docs/images/VisualDL/train_loss.png new file mode 100644 index 0000000..6697707 Binary files /dev/null and b/src/PaddleClas/docs/images/VisualDL/train_loss.png differ diff --git a/src/PaddleClas/docs/images/algorithm_introduction/fpgm.png b/src/PaddleClas/docs/images/algorithm_introduction/fpgm.png new file mode 100644 index 0000000..a773e8d Binary files /dev/null and b/src/PaddleClas/docs/images/algorithm_introduction/fpgm.png differ diff --git a/src/PaddleClas/docs/images/algorithm_introduction/quantization.jpg b/src/PaddleClas/docs/images/algorithm_introduction/quantization.jpg new file mode 100644 index 0000000..abb99b7 Binary files /dev/null and b/src/PaddleClas/docs/images/algorithm_introduction/quantization.jpg differ diff --git a/src/PaddleClas/docs/images/algorithm_introduction/quantization_formula.png b/src/PaddleClas/docs/images/algorithm_introduction/quantization_formula.png new file mode 100644 index 0000000..4d9dbe2 Binary files /dev/null and b/src/PaddleClas/docs/images/algorithm_introduction/quantization_formula.png differ diff --git a/src/PaddleClas/docs/images/algorithm_introduction/quantization_formula_slim.png b/src/PaddleClas/docs/images/algorithm_introduction/quantization_formula_slim.png new file mode 100644 index 0000000..989f6ac Binary files /dev/null and b/src/PaddleClas/docs/images/algorithm_introduction/quantization_formula_slim.png differ diff --git a/src/PaddleClas/docs/images/det/PaddleDetection_config.png b/src/PaddleClas/docs/images/det/PaddleDetection_config.png new file mode 100644 index 0000000..d18932b Binary files /dev/null and b/src/PaddleClas/docs/images/det/PaddleDetection_config.png differ diff --git a/src/PaddleClas/docs/images/det/pssdet.png b/src/PaddleClas/docs/images/det/pssdet.png new file mode 100644 index 0000000..e3b213e Binary files /dev/null and b/src/PaddleClas/docs/images/det/pssdet.png differ diff --git a/src/PaddleClas/docs/images/distillation/22k_1k_val_compare_w_sift.png b/src/PaddleClas/docs/images/distillation/22k_1k_val_compare_w_sift.png new file mode 100644 index 0000000..d97e6f9 Binary files /dev/null and b/src/PaddleClas/docs/images/distillation/22k_1k_val_compare_w_sift.png differ diff --git a/src/PaddleClas/docs/images/distillation/distillation_perform.png b/src/PaddleClas/docs/images/distillation/distillation_perform.png new file mode 100644 index 0000000..34cbacd Binary files /dev/null and b/src/PaddleClas/docs/images/distillation/distillation_perform.png differ diff --git a/src/PaddleClas/docs/images/distillation/distillation_perform_s.jpg b/src/PaddleClas/docs/images/distillation/distillation_perform_s.jpg new file mode 100644 index 0000000..181a490 Binary files /dev/null and b/src/PaddleClas/docs/images/distillation/distillation_perform_s.jpg differ diff --git a/src/PaddleClas/docs/images/distillation/ppcls_distillation.png b/src/PaddleClas/docs/images/distillation/ppcls_distillation.png new file mode 100644 index 0000000..d6cd7af Binary files /dev/null and b/src/PaddleClas/docs/images/distillation/ppcls_distillation.png differ diff --git a/src/PaddleClas/docs/images/distillation/ppcls_distillation_s.jpg b/src/PaddleClas/docs/images/distillation/ppcls_distillation_s.jpg new file mode 100644 index 0000000..95402d4 Binary files /dev/null and b/src/PaddleClas/docs/images/distillation/ppcls_distillation_s.jpg differ diff --git a/src/PaddleClas/docs/images/drink_demo.gif b/src/PaddleClas/docs/images/drink_demo.gif new file mode 100644 index 0000000..599a525 Binary files /dev/null and b/src/PaddleClas/docs/images/drink_demo.gif differ diff --git a/src/PaddleClas/docs/images/faq/ACNetFusion.png b/src/PaddleClas/docs/images/faq/ACNetFusion.png new file mode 100644 index 0000000..7068d93 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/ACNetFusion.png differ diff --git a/src/PaddleClas/docs/images/faq/ACNetReParams.png b/src/PaddleClas/docs/images/faq/ACNetReParams.png new file mode 100644 index 0000000..91dae1b Binary files /dev/null and b/src/PaddleClas/docs/images/faq/ACNetReParams.png differ diff --git a/src/PaddleClas/docs/images/faq/DeployedACNet.png b/src/PaddleClas/docs/images/faq/DeployedACNet.png new file mode 100644 index 0000000..e4900e0 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/DeployedACNet.png differ diff --git a/src/PaddleClas/docs/images/faq/HRNet.png b/src/PaddleClas/docs/images/faq/HRNet.png new file mode 100644 index 0000000..87d2992 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/HRNet.png differ diff --git a/src/PaddleClas/docs/images/faq/HRNet_block.png b/src/PaddleClas/docs/images/faq/HRNet_block.png new file mode 100644 index 0000000..03c244a Binary files /dev/null and b/src/PaddleClas/docs/images/faq/HRNet_block.png differ diff --git a/src/PaddleClas/docs/images/faq/HorizontalKernel.png b/src/PaddleClas/docs/images/faq/HorizontalKernel.png new file mode 100644 index 0000000..eb00792 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/HorizontalKernel.png differ diff --git a/src/PaddleClas/docs/images/faq/MemoryOccupation.png b/src/PaddleClas/docs/images/faq/MemoryOccupation.png new file mode 100644 index 0000000..d4dec8d Binary files /dev/null and b/src/PaddleClas/docs/images/faq/MemoryOccupation.png differ diff --git a/src/PaddleClas/docs/images/faq/RepVGG.png b/src/PaddleClas/docs/images/faq/RepVGG.png new file mode 100644 index 0000000..5c09ae7 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/RepVGG.png differ diff --git a/src/PaddleClas/docs/images/faq/RepVGGReParams.png b/src/PaddleClas/docs/images/faq/RepVGGReParams.png new file mode 100644 index 0000000..5b95156 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/RepVGGReParams.png differ diff --git a/src/PaddleClas/docs/images/faq/ResNet_vabcd_structure.png b/src/PaddleClas/docs/images/faq/ResNet_vabcd_structure.png new file mode 100644 index 0000000..8aacb67 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/ResNet_vabcd_structure.png differ diff --git a/src/PaddleClas/docs/images/faq/SE_structure.png b/src/PaddleClas/docs/images/faq/SE_structure.png new file mode 100644 index 0000000..1d96ec3 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/SE_structure.png differ diff --git a/src/PaddleClas/docs/images/faq/TrainingtimeACNet.png b/src/PaddleClas/docs/images/faq/TrainingtimeACNet.png new file mode 100644 index 0000000..1d8d0a1 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/TrainingtimeACNet.png differ diff --git a/src/PaddleClas/docs/images/faq/Transformer_input.png b/src/PaddleClas/docs/images/faq/Transformer_input.png new file mode 100644 index 0000000..977f092 Binary files /dev/null and b/src/PaddleClas/docs/images/faq/Transformer_input.png differ diff --git a/src/PaddleClas/docs/images/faq/ViT.png b/src/PaddleClas/docs/images/faq/ViT.png new file mode 100644 index 0000000..41a36ef Binary files /dev/null and b/src/PaddleClas/docs/images/faq/ViT.png differ diff --git a/src/PaddleClas/docs/images/faq/ViT_structure.png b/src/PaddleClas/docs/images/faq/ViT_structure.png new file mode 100644 index 0000000..25ebe9f Binary files /dev/null and b/src/PaddleClas/docs/images/faq/ViT_structure.png differ diff --git a/src/PaddleClas/docs/images/faq/momentum.jpeg b/src/PaddleClas/docs/images/faq/momentum.jpeg new file mode 100644 index 0000000..cb3cb0b Binary files /dev/null and b/src/PaddleClas/docs/images/faq/momentum.jpeg differ diff --git a/src/PaddleClas/docs/images/feature_compare.png b/src/PaddleClas/docs/images/feature_compare.png new file mode 100644 index 0000000..e553231 Binary files /dev/null and b/src/PaddleClas/docs/images/feature_compare.png differ diff --git a/src/PaddleClas/docs/images/feature_extraction_framework.png b/src/PaddleClas/docs/images/feature_extraction_framework.png new file mode 100644 index 0000000..5fafdbd Binary files /dev/null and b/src/PaddleClas/docs/images/feature_extraction_framework.png differ diff --git a/src/PaddleClas/docs/images/feature_extraction_framework_en.png b/src/PaddleClas/docs/images/feature_extraction_framework_en.png new file mode 100644 index 0000000..653d2a8 Binary files /dev/null and b/src/PaddleClas/docs/images/feature_extraction_framework_en.png differ diff --git a/src/PaddleClas/docs/images/feature_extraction_output.png b/src/PaddleClas/docs/images/feature_extraction_output.png new file mode 100644 index 0000000..8e8d0dd Binary files /dev/null and b/src/PaddleClas/docs/images/feature_extraction_output.png differ diff --git a/src/PaddleClas/docs/images/feature_maps/feature_visualization_input.jpg b/src/PaddleClas/docs/images/feature_maps/feature_visualization_input.jpg new file mode 100644 index 0000000..6eb5b71 Binary files /dev/null and b/src/PaddleClas/docs/images/feature_maps/feature_visualization_input.jpg differ diff --git a/src/PaddleClas/docs/images/feature_maps/feature_visualization_output.jpg b/src/PaddleClas/docs/images/feature_maps/feature_visualization_output.jpg new file mode 100644 index 0000000..6d7d2a6 Binary files /dev/null and b/src/PaddleClas/docs/images/feature_maps/feature_visualization_output.jpg differ diff --git a/src/PaddleClas/docs/images/icartoon1.png b/src/PaddleClas/docs/images/icartoon1.png new file mode 100644 index 0000000..c52989f Binary files /dev/null and b/src/PaddleClas/docs/images/icartoon1.png differ diff --git a/src/PaddleClas/docs/images/icartoon2.jpeg b/src/PaddleClas/docs/images/icartoon2.jpeg new file mode 100644 index 0000000..1601126 Binary files /dev/null and b/src/PaddleClas/docs/images/icartoon2.jpeg differ diff --git a/src/PaddleClas/docs/images/image_aug/gridmask-0.png b/src/PaddleClas/docs/images/image_aug/gridmask-0.png new file mode 100644 index 0000000..2d6997a Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/gridmask-0.png differ diff --git a/src/PaddleClas/docs/images/image_aug/hide-and-seek-visual.png b/src/PaddleClas/docs/images/image_aug/hide-and-seek-visual.png new file mode 100644 index 0000000..7f25576 Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/hide-and-seek-visual.png differ diff --git a/src/PaddleClas/docs/images/image_aug/image_aug_samples.png b/src/PaddleClas/docs/images/image_aug/image_aug_samples.png new file mode 100644 index 0000000..0861fcd Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/image_aug_samples.png differ diff --git a/src/PaddleClas/docs/images/image_aug/image_aug_samples_s.jpg b/src/PaddleClas/docs/images/image_aug/image_aug_samples_s.jpg new file mode 100644 index 0000000..aad144a Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/image_aug_samples_s.jpg differ diff --git a/src/PaddleClas/docs/images/image_aug/image_aug_samples_s_en.jpg b/src/PaddleClas/docs/images/image_aug/image_aug_samples_s_en.jpg new file mode 100644 index 0000000..2cd1a3a Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/image_aug_samples_s_en.jpg differ diff --git a/src/PaddleClas/docs/images/image_aug/main_image_aug.png b/src/PaddleClas/docs/images/image_aug/main_image_aug.png new file mode 100644 index 0000000..82a517a Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/main_image_aug.png differ diff --git a/src/PaddleClas/docs/images/image_aug/main_image_aug_s.jpg b/src/PaddleClas/docs/images/image_aug/main_image_aug_s.jpg new file mode 100644 index 0000000..efaa27c Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/main_image_aug_s.jpg differ diff --git a/src/PaddleClas/docs/images/image_aug/test_autoaugment.jpeg b/src/PaddleClas/docs/images/image_aug/test_autoaugment.jpeg new file mode 100644 index 0000000..207017f Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_autoaugment.jpeg differ diff --git a/src/PaddleClas/docs/images/image_aug/test_baseline.jpeg b/src/PaddleClas/docs/images/image_aug/test_baseline.jpeg new file mode 100644 index 0000000..3fa2d3a Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_baseline.jpeg differ diff --git a/src/PaddleClas/docs/images/image_aug/test_cutmix.png b/src/PaddleClas/docs/images/image_aug/test_cutmix.png new file mode 100644 index 0000000..9a396be Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_cutmix.png differ diff --git a/src/PaddleClas/docs/images/image_aug/test_cutout.jpeg b/src/PaddleClas/docs/images/image_aug/test_cutout.jpeg new file mode 100644 index 0000000..ad18f9d Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_cutout.jpeg differ diff --git a/src/PaddleClas/docs/images/image_aug/test_gridmask.jpeg b/src/PaddleClas/docs/images/image_aug/test_gridmask.jpeg new file mode 100644 index 0000000..698c793 Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_gridmask.jpeg differ diff --git a/src/PaddleClas/docs/images/image_aug/test_hideandseek.jpeg b/src/PaddleClas/docs/images/image_aug/test_hideandseek.jpeg new file mode 100644 index 0000000..55f1ada Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_hideandseek.jpeg differ diff --git a/src/PaddleClas/docs/images/image_aug/test_mixup.png b/src/PaddleClas/docs/images/image_aug/test_mixup.png new file mode 100644 index 0000000..e5d2807 Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_mixup.png differ diff --git a/src/PaddleClas/docs/images/image_aug/test_randaugment.jpeg b/src/PaddleClas/docs/images/image_aug/test_randaugment.jpeg new file mode 100644 index 0000000..7126f05 Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_randaugment.jpeg differ diff --git a/src/PaddleClas/docs/images/image_aug/test_randomerassing.jpeg b/src/PaddleClas/docs/images/image_aug/test_randomerassing.jpeg new file mode 100644 index 0000000..ff31bc2 Binary files /dev/null and b/src/PaddleClas/docs/images/image_aug/test_randomerassing.jpeg differ diff --git a/src/PaddleClas/docs/images/inference_deployment/cpp_infer_result.png b/src/PaddleClas/docs/images/inference_deployment/cpp_infer_result.png new file mode 100644 index 0000000..fe9de1a Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/cpp_infer_result.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/lite_demo_result.png b/src/PaddleClas/docs/images/inference_deployment/lite_demo_result.png new file mode 100644 index 0000000..b778f15 Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/lite_demo_result.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/vs2019_step1.png b/src/PaddleClas/docs/images/inference_deployment/vs2019_step1.png new file mode 100644 index 0000000..58fdf2f Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/vs2019_step1.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/vs2019_step2.png b/src/PaddleClas/docs/images/inference_deployment/vs2019_step2.png new file mode 100644 index 0000000..f1b5fd0 Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/vs2019_step2.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/vs2019_step3.png b/src/PaddleClas/docs/images/inference_deployment/vs2019_step3.png new file mode 100644 index 0000000..cb1b2ae Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/vs2019_step3.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/vs2019_step4.png b/src/PaddleClas/docs/images/inference_deployment/vs2019_step4.png new file mode 100644 index 0000000..5fc0408 Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/vs2019_step4.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/vs2019_step5.png b/src/PaddleClas/docs/images/inference_deployment/vs2019_step5.png new file mode 100644 index 0000000..fbb2e4c Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/vs2019_step5.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/vs2019_step6.png b/src/PaddleClas/docs/images/inference_deployment/vs2019_step6.png new file mode 100644 index 0000000..86a8039 Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/vs2019_step6.png differ diff --git a/src/PaddleClas/docs/images/inference_deployment/whl_demo.jpg b/src/PaddleClas/docs/images/inference_deployment/whl_demo.jpg new file mode 100644 index 0000000..cc7bff4 Binary files /dev/null and b/src/PaddleClas/docs/images/inference_deployment/whl_demo.jpg differ diff --git a/src/PaddleClas/docs/images/joinus.png b/src/PaddleClas/docs/images/joinus.png new file mode 100644 index 0000000..5fddb4a Binary files /dev/null and b/src/PaddleClas/docs/images/joinus.png differ diff --git a/src/PaddleClas/docs/images/logo.png b/src/PaddleClas/docs/images/logo.png new file mode 100644 index 0000000..bc1135a Binary files /dev/null and b/src/PaddleClas/docs/images/logo.png differ diff --git a/src/PaddleClas/docs/images/logo/logodet3k.jpg b/src/PaddleClas/docs/images/logo/logodet3k.jpg new file mode 100644 index 0000000..4c3e719 Binary files /dev/null and b/src/PaddleClas/docs/images/logo/logodet3k.jpg differ diff --git a/src/PaddleClas/docs/images/main_features.png b/src/PaddleClas/docs/images/main_features.png new file mode 100644 index 0000000..d663c99 Binary files /dev/null and b/src/PaddleClas/docs/images/main_features.png differ diff --git a/src/PaddleClas/docs/images/main_features_s.png b/src/PaddleClas/docs/images/main_features_s.png new file mode 100644 index 0000000..266fa22 Binary files /dev/null and b/src/PaddleClas/docs/images/main_features_s.png differ diff --git a/src/PaddleClas/docs/images/main_features_s_en.png b/src/PaddleClas/docs/images/main_features_s_en.png new file mode 100644 index 0000000..8f038fd Binary files /dev/null and b/src/PaddleClas/docs/images/main_features_s_en.png differ diff --git a/src/PaddleClas/docs/images/mainpage/cartoon_vis_en_v1.0.gif b/src/PaddleClas/docs/images/mainpage/cartoon_vis_en_v1.0.gif new file mode 100644 index 0000000..9450f09 Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/cartoon_vis_en_v1.0.gif differ diff --git a/src/PaddleClas/docs/images/mainpage/cartoon_vis_v1.0.gif b/src/PaddleClas/docs/images/mainpage/cartoon_vis_v1.0.gif new file mode 100644 index 0000000..86b3f8a Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/cartoon_vis_v1.0.gif differ diff --git a/src/PaddleClas/docs/images/mainpage/logo_vis_en_v1.0.gif b/src/PaddleClas/docs/images/mainpage/logo_vis_en_v1.0.gif new file mode 100644 index 0000000..a3fceb2 Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/logo_vis_en_v1.0.gif differ diff --git a/src/PaddleClas/docs/images/mainpage/logo_vis_v1.0.gif b/src/PaddleClas/docs/images/mainpage/logo_vis_v1.0.gif new file mode 100644 index 0000000..d8062ac Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/logo_vis_v1.0.gif differ diff --git a/src/PaddleClas/docs/images/mainpage/product_vis_en_v1.0.gif b/src/PaddleClas/docs/images/mainpage/product_vis_en_v1.0.gif new file mode 100644 index 0000000..eec4487 Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/product_vis_en_v1.0.gif differ diff --git a/src/PaddleClas/docs/images/mainpage/product_vis_v1.0.gif b/src/PaddleClas/docs/images/mainpage/product_vis_v1.0.gif new file mode 100644 index 0000000..3042aca Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/product_vis_v1.0.gif differ diff --git a/src/PaddleClas/docs/images/mainpage/recognition_pipeline_en.png b/src/PaddleClas/docs/images/mainpage/recognition_pipeline_en.png new file mode 100644 index 0000000..854fe55 Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/recognition_pipeline_en.png differ diff --git a/src/PaddleClas/docs/images/mainpage/vehicle_vis_en_v1.0.gif b/src/PaddleClas/docs/images/mainpage/vehicle_vis_en_v1.0.gif new file mode 100644 index 0000000..75e218e Binary files /dev/null and b/src/PaddleClas/docs/images/mainpage/vehicle_vis_en_v1.0.gif differ diff --git a/src/PaddleClas/docs/images/ml_illustration.jpg b/src/PaddleClas/docs/images/ml_illustration.jpg new file mode 100644 index 0000000..69dced9 Binary files /dev/null and b/src/PaddleClas/docs/images/ml_illustration.jpg differ diff --git a/src/PaddleClas/docs/images/ml_pipeline.jpg b/src/PaddleClas/docs/images/ml_pipeline.jpg new file mode 100644 index 0000000..cac6508 Binary files /dev/null and b/src/PaddleClas/docs/images/ml_pipeline.jpg differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs1.EfficientNet.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs1.EfficientNet.png new file mode 100644 index 0000000..1c87d71 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs1.EfficientNet.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.DPN.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.DPN.png new file mode 100644 index 0000000..1eb3939 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.DPN.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.HRNet.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.HRNet.png new file mode 100644 index 0000000..f21d63c Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.HRNet.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.Inception.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.Inception.png new file mode 100644 index 0000000..8095a3c Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.Inception.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.ResNet.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.ResNet.png new file mode 100644 index 0000000..dd25276 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.ResNet.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.SeResNeXt.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.SeResNeXt.png new file mode 100644 index 0000000..960e62a Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp16.bs4.SeResNeXt.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs1.EfficientNet.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs1.EfficientNet.png new file mode 100644 index 0000000..395a32f Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs1.EfficientNet.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs1.main_fps_top1.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs1.main_fps_top1.png new file mode 100644 index 0000000..0497028 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs1.main_fps_top1.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.flops.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.flops.png new file mode 100644 index 0000000..24aabf8 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.flops.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.params.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.params.png new file mode 100644 index 0000000..689e73d Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.params.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.png new file mode 100644 index 0000000..dc3922d Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.DPN.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.flops.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.flops.png new file mode 100644 index 0000000..deacfac Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.flops.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.params.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.params.png new file mode 100644 index 0000000..7177bbc Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.params.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.flops.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.flops.png new file mode 100644 index 0000000..062ecd7 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.flops.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.params.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.params.png new file mode 100644 index 0000000..4bb3f76 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.params.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.png new file mode 100644 index 0000000..3905f0b Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.HRNet.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.flops.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.flops.png new file mode 100644 index 0000000..6fdc94b Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.flops.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.params.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.params.png new file mode 100644 index 0000000..25a5d16 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.params.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.png new file mode 100644 index 0000000..7ef4f33 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.Inception.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.flops.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.flops.png new file mode 100644 index 0000000..57e0303 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.flops.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.params.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.params.png new file mode 100644 index 0000000..3440e3a Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.params.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.png new file mode 100644 index 0000000..8db3c2d Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.ResNet.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.flops.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.flops.png new file mode 100644 index 0000000..4e6fa84 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.flops.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.params.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.params.png new file mode 100644 index 0000000..afb66bd Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.params.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.png new file mode 100644 index 0000000..10cdd0a Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.main_fps_top1.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.main_fps_top1.png new file mode 100644 index 0000000..9750c19 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.main_fps_top1.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.flops.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.flops.png new file mode 100644 index 0000000..8c1be3a Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.flops.png differ diff --git a/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.params.png b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.params.png new file mode 100644 index 0000000..a41a325 Binary files /dev/null and b/src/PaddleClas/docs/images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.params.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.DPN.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.DPN.png new file mode 100644 index 0000000..10f542b Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.DPN.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.EfficientNet.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.EfficientNet.png new file mode 100644 index 0000000..0491ff3 Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.EfficientNet.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.HRNet.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.HRNet.png new file mode 100644 index 0000000..284088e Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.HRNet.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.Inception.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.Inception.png new file mode 100644 index 0000000..0e0e42e Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.Inception.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.ResNet.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.ResNet.png new file mode 100644 index 0000000..2332791 Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.ResNet.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.SeResNeXt.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.SeResNeXt.png new file mode 100644 index 0000000..610c846 Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.SeResNeXt.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.main_fps_top1_s.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.main_fps_top1_s.png new file mode 100644 index 0000000..ab7f3eb Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.main_fps_top1_s.png differ diff --git a/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.visiontransformer.png b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.visiontransformer.png new file mode 100644 index 0000000..5009619 Binary files /dev/null and b/src/PaddleClas/docs/images/models/V100_benchmark/v100.fp32.bs1.visiontransformer.png differ diff --git a/src/PaddleClas/docs/images/models/lr_decay.jpeg b/src/PaddleClas/docs/images/models/lr_decay.jpeg new file mode 100644 index 0000000..b482b35 Binary files /dev/null and b/src/PaddleClas/docs/images/models/lr_decay.jpeg differ diff --git a/src/PaddleClas/docs/images/models/mobile_arm_storage.png b/src/PaddleClas/docs/images/models/mobile_arm_storage.png new file mode 100644 index 0000000..07e1f4f Binary files /dev/null and b/src/PaddleClas/docs/images/models/mobile_arm_storage.png differ diff --git a/src/PaddleClas/docs/images/models/mobile_arm_top1.png b/src/PaddleClas/docs/images/models/mobile_arm_top1.png new file mode 100644 index 0000000..9156764 Binary files /dev/null and b/src/PaddleClas/docs/images/models/mobile_arm_top1.png differ diff --git a/src/PaddleClas/docs/images/paddlepaddle.jpg b/src/PaddleClas/docs/images/paddlepaddle.jpg new file mode 100644 index 0000000..da4cfb0 Binary files /dev/null and b/src/PaddleClas/docs/images/paddlepaddle.jpg differ diff --git a/src/PaddleClas/docs/images/ppshitu_lite_demo.png b/src/PaddleClas/docs/images/ppshitu_lite_demo.png new file mode 100644 index 0000000..c9b1766 Binary files /dev/null and b/src/PaddleClas/docs/images/ppshitu_lite_demo.png differ diff --git a/src/PaddleClas/docs/images/product/aliproduct.png b/src/PaddleClas/docs/images/product/aliproduct.png new file mode 100644 index 0000000..b57f96c Binary files /dev/null and b/src/PaddleClas/docs/images/product/aliproduct.png differ diff --git a/src/PaddleClas/docs/images/product/inshop.png b/src/PaddleClas/docs/images/product/inshop.png new file mode 100644 index 0000000..e874fdb Binary files /dev/null and b/src/PaddleClas/docs/images/product/inshop.png differ diff --git a/src/PaddleClas/docs/images/qq_group.png b/src/PaddleClas/docs/images/qq_group.png new file mode 100644 index 0000000..dad54c1 Binary files /dev/null and b/src/PaddleClas/docs/images/qq_group.png differ diff --git a/src/PaddleClas/docs/images/quick_start/Examples-Flower-102.png b/src/PaddleClas/docs/images/quick_start/Examples-Flower-102.png new file mode 100644 index 0000000..28adc5a Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/Examples-Flower-102.png differ diff --git a/src/PaddleClas/docs/images/quick_start/all_acc.png b/src/PaddleClas/docs/images/quick_start/all_acc.png new file mode 100644 index 0000000..cf9938c Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/all_acc.png differ diff --git a/src/PaddleClas/docs/images/quick_start/community/001_fork.png b/src/PaddleClas/docs/images/quick_start/community/001_fork.png new file mode 100644 index 0000000..a610381 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/community/001_fork.png differ diff --git a/src/PaddleClas/docs/images/quick_start/community/002_clone.png b/src/PaddleClas/docs/images/quick_start/community/002_clone.png new file mode 100644 index 0000000..f622634 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/community/002_clone.png differ diff --git a/src/PaddleClas/docs/images/quick_start/community/003_precommit_pass.png b/src/PaddleClas/docs/images/quick_start/community/003_precommit_pass.png new file mode 100644 index 0000000..067fb75 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/community/003_precommit_pass.png differ diff --git a/src/PaddleClas/docs/images/quick_start/community/004_create_pr.png b/src/PaddleClas/docs/images/quick_start/community/004_create_pr.png new file mode 100644 index 0000000..9276629 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/community/004_create_pr.png differ diff --git a/src/PaddleClas/docs/images/quick_start/community/code_framework.png b/src/PaddleClas/docs/images/quick_start/community/code_framework.png new file mode 100644 index 0000000..bc738dc Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/community/code_framework.png differ diff --git a/src/PaddleClas/docs/images/quick_start/community/train_framework.png b/src/PaddleClas/docs/images/quick_start/community/train_framework.png new file mode 100644 index 0000000..3501e02 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/community/train_framework.png differ diff --git a/src/PaddleClas/docs/images/quick_start/flowers102/image_06739.jpg b/src/PaddleClas/docs/images/quick_start/flowers102/image_06739.jpg new file mode 100644 index 0000000..19a1ed6 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/flowers102/image_06739.jpg differ diff --git a/src/PaddleClas/docs/images/quick_start/flowers102/image_06767.jpg b/src/PaddleClas/docs/images/quick_start/flowers102/image_06767.jpg new file mode 100644 index 0000000..28fd2db Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/flowers102/image_06767.jpg differ diff --git a/src/PaddleClas/docs/images/quick_start/flowers102/image_06773.jpg b/src/PaddleClas/docs/images/quick_start/flowers102/image_06773.jpg new file mode 100644 index 0000000..f4fde55 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/flowers102/image_06773.jpg differ diff --git a/src/PaddleClas/docs/images/quick_start/r50_vd_acc.png b/src/PaddleClas/docs/images/quick_start/r50_vd_acc.png new file mode 100644 index 0000000..3e28b3a Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/r50_vd_acc.png differ diff --git a/src/PaddleClas/docs/images/quick_start/r50_vd_pretrained_acc.png b/src/PaddleClas/docs/images/quick_start/r50_vd_pretrained_acc.png new file mode 100644 index 0000000..2d77049 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/r50_vd_pretrained_acc.png differ diff --git a/src/PaddleClas/docs/images/quick_start/shitu_c++_result.png b/src/PaddleClas/docs/images/quick_start/shitu_c++_result.png new file mode 100644 index 0000000..f2cbee5 Binary files /dev/null and b/src/PaddleClas/docs/images/quick_start/shitu_c++_result.png differ diff --git a/src/PaddleClas/docs/images/recognition.gif b/src/PaddleClas/docs/images/recognition.gif new file mode 100644 index 0000000..cbe150b Binary files /dev/null and b/src/PaddleClas/docs/images/recognition.gif differ diff --git a/src/PaddleClas/docs/images/recognition/crop_example.png b/src/PaddleClas/docs/images/recognition/crop_example.png new file mode 100644 index 0000000..368c3e3 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/crop_example.png differ diff --git a/src/PaddleClas/docs/images/recognition/drink_data_demo/output/mosilian.jpeg b/src/PaddleClas/docs/images/recognition/drink_data_demo/output/mosilian.jpeg new file mode 100644 index 0000000..ca7f2db Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/drink_data_demo/output/mosilian.jpeg differ diff --git a/src/PaddleClas/docs/images/recognition/drink_data_demo/output/nongfu_spring.jpeg b/src/PaddleClas/docs/images/recognition/drink_data_demo/output/nongfu_spring.jpeg new file mode 100644 index 0000000..b6669ba Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/drink_data_demo/output/nongfu_spring.jpeg differ diff --git a/src/PaddleClas/docs/images/recognition/drink_data_demo/output/youjia.jpeg b/src/PaddleClas/docs/images/recognition/drink_data_demo/output/youjia.jpeg new file mode 100644 index 0000000..a437915 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/drink_data_demo/output/youjia.jpeg differ diff --git a/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/mosilian.jpeg b/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/mosilian.jpeg new file mode 100644 index 0000000..22c6075 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/mosilian.jpeg differ diff --git a/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/nongfu_spring.jpeg b/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/nongfu_spring.jpeg new file mode 100644 index 0000000..0f4166d Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/nongfu_spring.jpeg differ diff --git a/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/youjia.jpeg b/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/youjia.jpeg new file mode 100644 index 0000000..2875a76 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/drink_data_demo/test_images/youjia.jpeg differ diff --git a/src/PaddleClas/docs/images/recognition/logo_demo/gallery/logo_gallery_demo.png b/src/PaddleClas/docs/images/recognition/logo_demo/gallery/logo_gallery_demo.png new file mode 100644 index 0000000..d748463 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/logo_demo/gallery/logo_gallery_demo.png differ diff --git a/src/PaddleClas/docs/images/recognition/logo_demo/query/logo_auxx-1.jpg b/src/PaddleClas/docs/images/recognition/logo_demo/query/logo_auxx-1.jpg new file mode 100644 index 0000000..90a9215 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/logo_demo/query/logo_auxx-1.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/logo_demo/query/logo_cola.jpg b/src/PaddleClas/docs/images/recognition/logo_demo/query/logo_cola.jpg new file mode 100644 index 0000000..97d0374 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/logo_demo/query/logo_cola.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/more_demo_images/cartoon.md b/src/PaddleClas/docs/images/recognition/more_demo_images/cartoon.md new file mode 100644 index 0000000..85dd1fa --- /dev/null +++ b/src/PaddleClas/docs/images/recognition/more_demo_images/cartoon.md @@ -0,0 +1,53 @@ +# 更多关于动漫人物图片识别的效果图请见下 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/images/recognition/more_demo_images/logo.md b/src/PaddleClas/docs/images/recognition/more_demo_images/logo.md new file mode 100644 index 0000000..03b992d --- /dev/null +++ b/src/PaddleClas/docs/images/recognition/more_demo_images/logo.md @@ -0,0 +1,65 @@ +# 更多关于商标图片识别的效果图请见下 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/images/recognition/more_demo_images/product.md b/src/PaddleClas/docs/images/recognition/more_demo_images/product.md new file mode 100644 index 0000000..7b33e27 --- /dev/null +++ b/src/PaddleClas/docs/images/recognition/more_demo_images/product.md @@ -0,0 +1,179 @@ +# 更多关于商品图片识别的效果图请见下 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/images/recognition/more_demo_images/vehicle.md b/src/PaddleClas/docs/images/recognition/more_demo_images/vehicle.md new file mode 100644 index 0000000..7fe4e92 --- /dev/null +++ b/src/PaddleClas/docs/images/recognition/more_demo_images/vehicle.md @@ -0,0 +1,33 @@ +# 更多关于车辆图片识别的效果图请见下 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/001.jpg b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/001.jpg new file mode 100644 index 0000000..586aef2 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/001.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/002.jpg b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/002.jpg new file mode 100644 index 0000000..8d4285d Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/002.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/003.jpg b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/003.jpg new file mode 100644 index 0000000..f4efa2b Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/003.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/004.jpg b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/004.jpg new file mode 100644 index 0000000..017aa82 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/004.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/005.jpg b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/005.jpg new file mode 100644 index 0000000..e7d6f88 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/005.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/006.jpg b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/006.jpg new file mode 100644 index 0000000..36a8e5d Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/gallery/anmuxi/006.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/query/anmuxi.jpg b/src/PaddleClas/docs/images/recognition/product_demo/query/anmuxi.jpg new file mode 100644 index 0000000..735ea5a Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/query/anmuxi.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/query/daoxiangcunjinzhubing_6.jpg b/src/PaddleClas/docs/images/recognition/product_demo/query/daoxiangcunjinzhubing_6.jpg new file mode 100644 index 0000000..fc64a95 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/query/daoxiangcunjinzhubing_6.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/result/anmuxi.jpg b/src/PaddleClas/docs/images/recognition/product_demo/result/anmuxi.jpg new file mode 100644 index 0000000..4b219e2 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/result/anmuxi.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/result/anmuxi_en.jpg b/src/PaddleClas/docs/images/recognition/product_demo/result/anmuxi_en.jpg new file mode 100644 index 0000000..b755a5f Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/result/anmuxi_en.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/result/daoxiangcunjinzhubing_6.jpg b/src/PaddleClas/docs/images/recognition/product_demo/result/daoxiangcunjinzhubing_6.jpg new file mode 100644 index 0000000..645ad6c Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/result/daoxiangcunjinzhubing_6.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/result/daoxiangcunjinzhubing_6_en.jpg b/src/PaddleClas/docs/images/recognition/product_demo/result/daoxiangcunjinzhubing_6_en.jpg new file mode 100644 index 0000000..102616b Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/result/daoxiangcunjinzhubing_6_en.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/wangzai.jpg b/src/PaddleClas/docs/images/recognition/product_demo/wangzai.jpg new file mode 100644 index 0000000..fe7aa77 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/wangzai.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/product_demo/wangzai_det_result.jpg b/src/PaddleClas/docs/images/recognition/product_demo/wangzai_det_result.jpg new file mode 100644 index 0000000..e7552d1 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/product_demo/wangzai_det_result.jpg differ diff --git a/src/PaddleClas/docs/images/recognition/rec_pipeline.png b/src/PaddleClas/docs/images/recognition/rec_pipeline.png new file mode 100644 index 0000000..f6a95e6 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/rec_pipeline.png differ diff --git a/src/PaddleClas/docs/images/recognition/vehicle/CompCars.png b/src/PaddleClas/docs/images/recognition/vehicle/CompCars.png new file mode 100644 index 0000000..021d8c4 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/vehicle/CompCars.png differ diff --git a/src/PaddleClas/docs/images/recognition/vehicle/cars.JPG b/src/PaddleClas/docs/images/recognition/vehicle/cars.JPG new file mode 100644 index 0000000..bb60246 Binary files /dev/null and b/src/PaddleClas/docs/images/recognition/vehicle/cars.JPG differ diff --git a/src/PaddleClas/docs/images/recognition_en.gif b/src/PaddleClas/docs/images/recognition_en.gif new file mode 100644 index 0000000..0adffea Binary files /dev/null and b/src/PaddleClas/docs/images/recognition_en.gif differ diff --git a/src/PaddleClas/docs/images/structure.jpg b/src/PaddleClas/docs/images/structure.jpg new file mode 100644 index 0000000..1d7f5a1 Binary files /dev/null and b/src/PaddleClas/docs/images/structure.jpg differ diff --git a/src/PaddleClas/docs/images/wx_group.png b/src/PaddleClas/docs/images/wx_group.png new file mode 100644 index 0000000..ab3bd91 Binary files /dev/null and b/src/PaddleClas/docs/images/wx_group.png differ diff --git a/src/PaddleClas/docs/make.bat b/src/PaddleClas/docs/make.bat new file mode 100644 index 0000000..061f32f --- /dev/null +++ b/src/PaddleClas/docs/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=source +set BUILDDIR=build + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.https://www.sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/src/PaddleClas/docs/requirements.txt b/src/PaddleClas/docs/requirements.txt new file mode 100644 index 0000000..a33c569 --- /dev/null +++ b/src/PaddleClas/docs/requirements.txt @@ -0,0 +1,4 @@ +recommonmark +Sphinx +sphinx-markdown-tables +sphinx-rtd-theme diff --git a/src/PaddleClas/docs/source/index.rst b/src/PaddleClas/docs/source/index.rst new file mode 100644 index 0000000..6ff5f7e --- /dev/null +++ b/src/PaddleClas/docs/source/index.rst @@ -0,0 +1,20 @@ +.. paddleclas documentation master file, created by + sphinx-quickstart on Wed Dec 15 16:13:23 2021. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +Welcome to paddleclas's documentation! +====================================== + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/src/PaddleClas/docs/zh_CN/Makefile b/src/PaddleClas/docs/zh_CN/Makefile new file mode 100644 index 0000000..7823a52 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/Makefile @@ -0,0 +1,22 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build + +#SOURCEDIR = source +BUILDDIR = build + + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/DataAugmentation.md b/src/PaddleClas/docs/zh_CN/advanced_tutorials/DataAugmentation.md new file mode 100644 index 0000000..9e5159a --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/DataAugmentation.md @@ -0,0 +1,331 @@ +# 数据增强分类实战 +--- + +本节将基于 ImageNet-1K 的数据集详细介绍数据增强实验，如果想快速体验此方法，可以参考 [**30 分钟玩转 PaddleClas（进阶版）**](../quick_start/quick_start_classification_professional.md)中基于 CIFAR100 的数据增强实验。如果想了解相关算法的内容，请参考[数据增强算法介绍](../algorithm_introduction/DataAugmentation.md)。 + + +## 目录 + +- [1. 参数配置](#1) + - [1.1 AutoAugment](#1.1) + - [1.2 RandAugment](#1.2) + - [1.3 TimmAutoAugment](#1.3) + - [1.4 Cutout](#1.4) + - [1.5 RandomErasing](#1.5) + - [1.6 HideAndSeek](#1.6) + - [1.7 GridMask](#1.7) + - [1.8 Mixup](#1.8) + - [1.9 Cutmix](#1.9) + - [1.10 Mixup 与 Cutmix 同时使用](#1.10) +- [2. 启动命令](#2) +- [3. 注意事项](#3) +- [4. 实验结果](#4) + + +## 1. 参数配置 + +由于不同的数据增强方式含有不同的超参数，为了便于理解和使用，我们在 `configs/DataAugment` 里分别列举了 8 种训练 ResNet50 的数据增强方式的参数配置文件，用户可以在 `tools/run.sh` 里直接替换配置文件的路径即可使用。此处分别挑选了图像变换、图像裁剪、图像混叠中的一个示例展示，其他参数配置用户可以自查配置文件。 + + +### 1.1 AutoAugment + +`AotoAugment` 的图像增广方式的配置如下。`AutoAugment` 是在 uint8 的数据格式上转换的，所以其处理过程应该放在归一化操作(`NormalizeImage`)之前。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - AutoAugment: + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + + +### 1.2 RandAugment + +`RandAugment` 的图像增广方式的配置如下，其中用户需要指定其中的参数 `num_layers` 与 `magnitude`，默认的数值分别是 `2` 和 `5`。`RandAugment` 是在 uint8 的数据格式上转换的，所以其处理过程应该放在归一化操作(`NormalizeImage`)之前。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - RandAugment: + num_layers: 2 + magnitude: 5 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + + +### 1.3 TimmAutoAugment + +`TimmAutoAugment` 的图像增广方式的配置如下，其中用户需要指定其中的参数 `config_str`、`interpolation`、`img_size`，默认的数值分别是 `rand-m9-mstd0.5-inc1`、`bicubic`、`224`。`TimmAutoAugment` 是在 uint8 的数据格式上转换的，所以其处理过程应该放在归一化操作(`NormalizeImage`)之前。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - TimmAutoAugment: + config_str: rand-m9-mstd0.5-inc1 + interpolation: bicubic + img_size: 224 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + + +### 1.4 Cutout + +`Cutout` 的图像增广方式的配置如下，其中用户需要指定其中的参数 `n_holes` 与 `length`，默认的数值分别是 `1` 和 `112`。类似其他图像裁剪类的数据增强方式，`Cutout` 既可以在 uint8 格式的数据上操作，也可以在归一化)(`NormalizeImage`)后的数据上操作，此处给出的是在归一化后的操作。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - Cutout: + n_holes: 1 + length: 112 +``` + + +### 1.5 RandomErasing + +`RandomErasing` 的图像增广方式的配置如下，其中用户需要指定其中的参数 `EPSILON`、`sl`、`sh`、`r1`、`attempt`、`use_log_aspect`、`mode`，默认的数值分别是 `0.25`、`0.02`、`1.0/3.0`、`0.3`、`10`、`True`、`pixel`。类似其他图像裁剪类的数据增强方式，`RandomErasing` 既可以在 uint8 格式的数据上操作，也可以在归一化(`NormalizeImage`)后的数据上操作，此处给出的是在归一化后的操作。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - RandomErasing: + EPSILON: 0.25 + sl: 0.02 + sh: 1.0/3.0 + r1: 0.3 + attempt: 10 + use_log_aspect: True + mode: pixel +``` + + +### 1.6 HideAndSeek + +`HideAndSeek` 的图像增广方式的配置如下。类似其他图像裁剪类的数据增强方式，`HideAndSeek` 既可以在 uint8 格式的数据上操作，也可以在归一化(`NormalizeImage`)后的数据上操作，此处给出的是在归一化后的操作。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - HideAndSeek: +``` + + + +### 1.7 GridMask + +`GridMask` 的图像增广方式的配置如下,其中用户需要指定其中的参数 `d1`、`d2`、`rotate`、`ratio`、`mode`, 默认的数值分别是 `96`、`224`、`1`、`0.5`、`0`。类似其他图像裁剪类的数据增强方式，`GridMask` 既可以在 uint8 格式的数据上操作，也可以在归一化(`NormalizeImage`)后的数据上操作，此处给出的是在归一化后的操作。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - GridMask: + d1: 96 + d2: 224 + rotate: 1 + ratio: 0.5 + mode: 0 +``` + + +### 1.8 Mixup + +`Mixup` 的图像增广方式的配置如下，其中用户需要指定其中的参数 `alpha`，默认的数值是 `0.2`。类似其他图像混合类的数据增强方式，`Mixup` 是在图像做完数据处理后将每个 batch 内的数据做图像混叠，将混叠后的图像和标签输入网络中训练，所以其是在图像数据处理（图像变换、图像裁剪）后操作。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + batch_transform_ops: + - MixupOperator: + alpha: 0.2 +``` + + +### 1.9 Cutmix + +`Cutmix` 的图像增广方式的配置如下，其中用户需要指定其中的参数 `alpha`，默认的数值是 `0.2`。类似其他图像混合类的数据增强方式，`Cutmix` 是在图像做完数据处理后将每个 batch 内的数据做图像混叠，将混叠后的图像和标签输入网络中训练，所以其是在图像数据处理（图像变换、图像裁剪）后操作。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + batch_transform_ops: + - CutmixOperator: + alpha: 0.2 +``` + + +### 1.10 Mixup 与 Cutmix 同时使用 + +`Mixup` 与 `Cutmix` 同时使用的配置如下，其中用户需要指定额外的参数 `prob`，该参数控制不同数据增强的概率，默认为 `0.5`。 + +```yaml + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - OpSampler: + MixupOperator: + alpha: 0.8 + prob: 0.5 + CutmixOperator: + alpha: 1.0 + prob: 0.5 +``` + + +## 2. 启动命令 + +当用户配置完训练环境后，类似于训练其他分类任务，只需要将 `tools/train.sh` 中的配置文件替换成为相应的数据增强方式的配置文件即可。 + +其中 `train.sh` 中的内容如下： + +```bash + +python3 -m paddle.distributed.launch \ + --selected_gpus="0,1,2,3" \ + --log_dir=ResNet50_Cutout \ + tools/train.py \ + -c ./ppcls/configs/ImageNet/DataAugment/ResNet50_Cutout.yaml +``` + +运行 `train.sh`： + +```bash +sh tools/train.sh +``` + + +## 3. 注意事项 + +* 由于图像混叠时需对 label 进行混叠，无法计算训练数据的准确率，所以在训练过程中没有打印训练准确率。 + +* 在使用数据增强后，由于训练数据更难，所以训练损失函数可能较大，训练集的准确率相对较低，但其有拥更好的泛化能力，所以验证集的准确率相对较高。 + +* 在使用数据增强后，模型可能会趋于欠拟合状态，建议可以适当的调小 `l2_decay` 的值来获得更高的验证集准确率。 + +* 几乎每一类图像增强均含有超参数，我们只提供了基于 ImageNet-1k 的超参数，其他数据集需要用户自己调试超参数，具体超参数的含义用户可以阅读相关的论文，调试方法也可以参考[训练技巧](../models_training/train_strategy.md)。 + + +## 4. 实验结果 + +基于 PaddleClas，在 ImageNet1k 数据集上的分类精度如下。 + +| 模型 | 初始学习率策略 | l2 decay | batch size | epoch | 数据变化策略 | Top1 Acc | 论文中结论 | +|-------------|------------------|--------------|------------|-------|----------------|------------|----| +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | 标准变换 | 0.7731 | - | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | AutoAugment | 0.7795 | 0.7763 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | mixup | 0.7828 | 0.7790 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | cutmix | 0.7839 | 0.7860 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | cutout | 0.7801 | - | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | gridmask | 0.7785 | 0.7790 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | random-augment | 0.7770 | 0.7760 | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | random erasing | 0.7791 | - | +| ResNet50 | 0.1/cosine_decay | 0.0001 | 256 | 300 | hide and seek | 0.7743 | 0.7720 | + +**注意**： +* 在这里的实验中，为了便于对比，我们将 l2 decay 固定设置为 1e-4，在实际使用中，我们推荐尝试使用更小的 l2 decay。结合数据增强，我们发现将 l2 decay 由 1e-4 减小为 7e-5 均能带来至少 0.3~0.5% 的精度提升。 +* 我们目前尚未对不同策略进行组合并验证效果，这一块后续我们会开展更多的对比实验，敬请期待。 diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/code_overview.md b/src/PaddleClas/docs/zh_CN/advanced_tutorials/code_overview.md new file mode 100644 index 0000000..ef4e503 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/code_overview.md @@ -0,0 +1,259 @@ +# PaddleClas 代码解析 + +## 目录 + +- [1. 整体代码和目录概览](#1) +- [2. 训练模块定义](#2) + - [2.1 数据](#2.1) + - [2.2 模型结构](#2.2) + - [2.3 损失函数](#2.3) + - [2.4 优化器和学习率衰减、权重衰减策略](#2.4) + - [2.5 训练时评估](#2.5) + - [2.6 模型存储](#2.6) + - [2.7 模型裁剪与量化](#2.7) +- [3. 预测部署代码和方式](#3) + + +## 1. 整体代码和目录概览 + +PaddleClas 主要代码和目录结构如下 + +* benchmark: 文件夹下存放了一些 shell 脚本，主要是为了测试 PaddleClas 中不同模型的速度指标，如单卡训练速度指标、多卡训练速度指标等。 +* dataset：文件夹下存放数据集和用于处理数据集的脚本。脚本负责将数据集处理为适合 Dataloader 处理的格式。 +* deploy：部署核心代码，文件夹存放的是部署工具，支持 python/cpp inference、Hub Serveing、Paddle Lite、Slim 离线量化等多种部署方式。 +* ppcls：训练核心代码，文件夹下存放 PaddleClas 框架主体。配置文件、模型训练、评估、预测、动转静导出等具体代码实现均在这里。 +* tools：训练、评估、预测、模型动转静导出的入口函数和脚本均在该文件下。 +* requirements.txt 文件用于安装 PaddleClas 的依赖项。使用 pip 进行升级安装使用。 +* tests：PaddleClas 模型从训练到预测的全链路测试，验证各功能是否能够正常使用。 + + +## 2. 训练模块定义 + +深度学习模型训练模块，主要包含数据、模型结构、损失函数、优化器和学习率衰减、权重衰减策略等，以下一一解读。 + + +### 2.1 数据 + +对于有监督任务来说，训练数据一般包含原始数据及其标注。 +在基于单标签的图像分类任务中，原始数据指的是图像数据，而标注则是该图像数据所属的类别。 +PaddleClas 中，训练时需要提供标签文件，形式如下，每一行包含一条训练样本，分别表示图片路径和类别标签，用分隔符隔开（默认为空格）。 + +``` +train/n01440764/n01440764_10026.JPEG 0 +train/n01440764/n01440764_10027.JPEG 0 +``` + +在代码 `ppcls/data/dataloader/common_dataset.py` 中，包含 `CommonDataset` 类，继承自 `paddle.io.Dataset`， +该数据集类可以通过一个键值进行索引并获取指定样本。`ImageNetDataset`, `LogoDataset`, `CommonDataset` 等数据集类都继承自这个类别 + +对于读入的数据，需要通过数据转换，将原始的图像数据进行转换。训练时，标准的数据预处理包含：`DecodeImage`, `RandCropImage`, +`RandFlipImage`, `NormalizeImage`, `ToCHWImage`。 +在配置文件中体现如下，数据预处理主要包含在 `transforms` 字段中，以列表形式呈现，会按照顺序对数据依次做这些转换。 + +```yaml +DataLoader: + Train: + dataset: + name: ImageNetDataset + image_root: ./dataset/ILSVRC2012/ + cls_label_path: ./dataset/ILSVRC2012/train_list.txt + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - RandCropImage: + size: 224 + - RandFlipImage: + flip_code: 1 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + +PaddleClas 中也包含了 `AutoAugment`, `RandAugment` 等数据增广方法，也可以通过在配置文件中配置，从而添加到训练过程的数据预处理中。 +每个数据转换的方法均以类实现，方便迁移和复用，更多的数据处理具体实现过程可以参考 `ppcls/data/preprocess/ops/` 下的代码。 + +对于组成一个 batch 的数据，也可以使用 mixup 或者 cutmix 等方法进行数据增广。 +PaddleClas 中集成了 `MixupOperator`, `CutmixOperator`, `FmixOperator` 等基于 batch 的数据增广方法， +可以在配置文件中配置 mix 参数进行配置，更加具体的实现可以参考 `ppcls/data/preprocess/batch_ops/batch_operators.py` 。 + +图像分类中，数据后处理主要为 `argmax` 操作，在此不再赘述。 + + +### 2.2 模型结构 + +在配置文件中，模型结构定义如下 + +```yaml +Arch: + name: ResNet50 + class_num: 1000 + pretrained: False + use_ssld: False +``` + +`Arch.name` 表示模型名称，`Arch.pretrained` 表示是否添加预训练模型，`Arch.use_ssld` 表示是否使用基于 `SSLD` 知识蒸馏得到的预训练模型。 +所有的模型名称均在 `ppcls/arch/backbone/__init__.py` 中定义。 + +对应的，在 `ppcls/arch/__init__.py` 中，通过 `build_model` 方法创建模型对象。 + +```python +def build_model(config): + config = copy.deepcopy(config) + model_type = config.pop("name") + mod = importlib.import_module(__name__) + arch = getattr(mod, model_type)(**config) + return arch +``` + + +### 2.3 损失函数 + +PaddleClas 中，包含了 `CELoss`, `JSDivLoss`, `TripletLoss`, `CenterLoss` 等损失函数，均定义在 `ppcls/loss` 中。 + +在 `ppcls/loss/__init__.py` 文件中，使用 `CombinedLoss` 来构建及合并损失函数，不同训练策略中所需要的损失函数与计算方法不同，PaddleClas 在构建损失函数过程中，主要考虑了以下几个因素。 + +1. 是否使用 label smooth +2. 是否使用 mixup 或者 cutmix +3. 是否使用蒸馏方法进行训练 +4. 是否是训练 metric learning + + +用户可以在配置文件中指定损失函数的类型及权重，如在训练中添加 TripletLossV2，配置文件如下： + +```yaml +Loss: + Train: + - CELoss: + weight: 1.0 + - TripletLossV2: + weight: 1.0 + margin: 0.5 +``` + + +### 2.4 优化器和学习率衰减、权重衰减策略 + +图像分类任务中，`Momentum` 是一种比较常用的优化器，PaddleClas 中提供了 `Momentum` 、 `RMSProp`、`Adam` 及 `AdamW` 等几种优化器策略。 + +权重衰减策略是一种比较常用的正则化方法，主要用于防止模型过拟合。 PaddleClas 中提供了 `L1Decay` 和 `L2Decay` 两种权重衰减策略。 + +学习率衰减是图像分类任务中必不可少的精度提升训练方法，PaddleClas 目前支持 `Cosine`, `Piecewise`, `Linear` 等学习率衰减策略。 + +在配置文件中，优化器、权重衰减策略、学习率衰减策略可以通过以下的字段进行配置。 + +```yaml +Optimizer: + name: Momentum + momentum: 0.9 + lr: + name: Piecewise + learning_rate: 0.1 + decay_epochs: [30, 60, 90] + values: [0.1, 0.01, 0.001, 0.0001] + regularizer: + name: 'L2' + coeff: 0.0001 +``` + +在 `ppcls/optimizer/__init__.py` 中使用 `build_optimizer` 创建优化器和学习率对象。 + +```python +def build_optimizer(config, epochs, step_each_epoch, parameters): + config = copy.deepcopy(config) + # step1 build lr + lr = build_lr_scheduler(config.pop('lr'), epochs, step_each_epoch) + logger.debug("build lr ({}) success..".format(lr)) + # step2 build regularization + if 'regularizer' in config and config['regularizer'] is not None: + reg_config = config.pop('regularizer') + reg_name = reg_config.pop('name') + 'Decay' + reg = getattr(paddle.regularizer, reg_name)(**reg_config) + else: + reg = None + logger.debug("build regularizer ({}) success..".format(reg)) + # step3 build optimizer + optim_name = config.pop('name') + if 'clip_norm' in config: + clip_norm = config.pop('clip_norm') + grad_clip = paddle.nn.ClipGradByNorm(clip_norm=clip_norm) + else: + grad_clip = None + optim = getattr(optimizer, optim_name)(learning_rate=lr, + weight_decay=reg, + grad_clip=grad_clip, + **config)(parameters=parameters) + logger.debug("build optimizer ({}) success..".format(optim)) + return optim, lr +``` + + 不同优化器和权重衰减策略均以类的形式实现，具体实现可以参考文件 `ppcls/optimizer/optimizer.py`. + 不同的学习率衰减策略可以参考文件 `ppcls/optimizer/learning_rate.py` 。 + + +### 2.5 训练时评估 + +模型在训练的时候，可以设置模型保存的间隔，也可以选择每隔若干个 epoch 对验证集进行评估， +从而可以保存在验证集上精度最佳的模型。配置文件中，可以通过下面的字段进行配置。 + +```yaml +Global: + save_interval: 1 # 模型保存的 epoch 间隔 + eval_during_train: True # 是否进行训练时评估 + eval_interval: 1 # 评估的 epoch 间隔 +``` + + +### 2.6 模型存储 +模型存储是通过 Paddle 框架的 `paddle.save()` 函数实现的，存储的是模型的动态图版本，以字典的形式存储，便于继续训练。具体实现如下 + +```python +def save_model(program, model_path, epoch_id, prefix='ppcls'): + model_path = os.path.join(model_path, str(epoch_id)) + _mkdir_if_not_exist(model_path) + model_prefix = os.path.join(model_path, prefix) + paddle.static.save(program, model_prefix) + logger.info( + logger.coloring("Already save model in {}".format(model_path), "HEADER")) +``` + +在保存的时候有两点需要注意： +1. 只在 0 号节点上保存模型。否则多卡训练的时候，如果所有节点都保存模型到相同的路径， +2. 则多个节点写文件时可能会发生写文件冲突，导致最终保存的模型无法被正确加载。 +3. 优化器参数也需要存储，方便后续的加载断点进行训练。 + + + +### 2.7 模型裁剪与量化 +如果想对模型进行压缩训练，则通过下面字段进行配置 +1.模型裁剪： + +```yaml +Slim: + prune: + name: fpgm + pruned_ratio: 0.3 +``` + +2.模型量化： + +```yaml +Slim: + quant: + name: pact +``` + +训练方法详见模型[裁剪量化使用介绍](../advanced_tutorials/model_prune_quantization.md)， +算法介绍详见[裁剪量化算法介绍](../algorithm_introduction/model_prune_quantization.md)。 + + +## 3. 预测部署代码和方式 + +* 如果希望将对分类模型进行离线量化，可以参考 [模型量化裁剪教程](../advanced_tutorials/model_prune_quantization.md) 中离线量化部分。 +* 如果希望在服务端使用 python 进行部署，可以参考 [python inference 预测教程](../inference_deployment/python_deploy.md)。 +* 如果希望在服务端使用 cpp 进行部署，可以参考 [cpp inference 预测教程](../inference_deployment/cpp_deploy.md)。 +* 如果希望将分类模型部署为服务，可以参考 [hub serving 预测部署教程](../inference_deployment/paddle_hub_serving_deploy.md)。 +* 如果希望在移动端使用分类模型进行预测，可以参考 [PaddleLite 预测部署教程](../inference_deployment/paddle_lite_deploy.md)。 +* 如果希望使用 whl 包对分类模型进行预测，可以参考 [whl 包预测](../inference_deployment/whl_deploy.md)。 diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/how_to_contribute.md b/src/PaddleClas/docs/zh_CN/advanced_tutorials/how_to_contribute.md new file mode 100644 index 0000000..681a7c7 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/how_to_contribute.md @@ -0,0 +1,252 @@ +# PaddleClas 社区贡献指南 +------ + +## 目录 + +- [1. 如何贡献代码](#1) + - [1.1 PaddleClas 分支说明](#1.1) + - [1.2 PaddleClas 代码提交流程与规范](#1.2) + - [1.2.1 fork 和 clone 代码](#1.2.1) + - [1.2.2 和远程仓库建立连接](#1.2.2) + - [1.2.3 创建本地分支](#1.2.3) + - [1.2.4 使用 pre-commit 勾子](#1.2.4) + - [1.2.5 修改与提交代码](#1.2.5) + - [1.2.6 保持本地仓库最新](#1.2.6) + - [1.2.7 push 到远程仓库](#1.2.7) + - [1.2.8 提交 Pull Request](#1.2.8) + - [1.2.9 签署 CLA 协议和通过单元测试](#1.2.9) + - [1.2.10 删除分支](#1.2.10) + - [1.2.11 提交代码的一些约定](#1.2.11) +- [2. 总结](#2) +- [3. 参考文献](#3) + + +## 1. 如何贡献代码 + + +### 1.1 PaddleClas 分支说明 + +PaddleClas 未来将维护 2 种分支，分别为： + +* release/x.x 系列分支：为稳定的发行版本分支，会适时打 tag 发布版本，适配 Paddle 的 release 版本。当前最新的分支为 release/2.3 分支，是当前默认分支，适配 Paddle v2.1.0 。随着版本迭代， release/x.x 系列分支会越来越多，默认维护最新版本的 release 分支，前 1 个版本分支会修复 bug，其他的分支不再维护。 +* develop 分支：为开发分支，适配 Paddle 的 develop 版本，主要用于开发新功能。如果有同学需要进行二次开发，请选择 develop 分支。为了保证 develop 分支能在需要的时候拉出 release/x.x 分支， develop 分支的代码只能使用 Paddle 最新 release 分支中有效的 api 。也就是说，如果 Paddle develop 分支中开发了新的 api，但尚未出现在 release 分支代码中，那么请不要在 PaddleClas 中使用。除此之外，对于不涉及 api 的性能优化、参数调整、策略更新等，都可以正常进行开发。 + +PaddleClas 的历史分支，未来将不再维护。考虑到一些同学可能仍在使用，这些分支还会继续保留： + +* release/static 分支：这个分支曾用于静态图的开发与测试，目前兼容 >=1.7 版本的 Paddle 。如果有特殊需求，要适配旧版本的 Paddle，那还可以使用这个分支，但除了修复 bug 外不再更新代码。 +* dygraph-dev 分支：这个分支将不再维护，也不再接受新的代码，请使用的同学尽快迁移到 develop 分支。 + + +PaddleClas 欢迎大家向 repo 中积极贡献代码，下面给出一些贡献代码的基本流程。 + + +### 1.2 PaddleClas 代码提交流程与规范 + + +#### 1.2.1 fork 和 clone 代码 + +* 跳转到 [PaddleClas GitHub 首页](https://github.com/PaddlePaddle/PaddleClas)，然后单击 Fork 按钮，生成自己目录下的仓库，比如 `https://github.com/USERNAME/PaddleClas` 。 + + +![](../../images/quick_start/community/001_fork.png) + +* 将远程仓库 clone 到本地 + +```shell +# 拉取 develop 分支的代码 +git clone https://github.com/USERNAME/PaddleClas.git -b develop +cd PaddleClas +``` + +clone 的地址可以从下面获取 + +![](../../images/quick_start/community/002_clone.png) + + +#### 1.2.2 和远程仓库建立连接 + +首先通过 `git remote -v` 查看当前远程仓库的信息。 + +``` +origin https://github.com/USERNAME/PaddleClas.git (fetch) +origin https://github.com/USERNAME/PaddleClas.git (push) +``` + +上面的信息只包含了 clone 的远程仓库的信息，也就是自己用户名下的 PaddleClas，接下来我们创建一个原始 PaddleClas 仓库的远程主机，命名为 upstream 。 + +```shell +git remote add upstream https://github.com/PaddlePaddle/PaddleClas.git +``` + +使用 `git remote -v` 查看当前远程仓库的信息，输出如下，发现包括了 origin 和 upstream 2 个远程仓库。 + +``` +origin https://github.com/USERNAME/PaddleClas.git (fetch) +origin https://github.com/USERNAME/PaddleClas.git (push) +upstream https://github.com/PaddlePaddle/PaddleClas.git (fetch) +upstream https://github.com/PaddlePaddle/PaddleClas.git (push) +``` + +这主要是为了后续在提交 pull request(PR)时，始终保持本地仓库最新。 + + +#### 1.2.3 创建本地分支 + +可以基于当前分支创建新的本地分支，命令如下。 + +```shell +git checkout -b new_branch +``` + +也可以基于远程或者上游的分支创建新的分支，命令如下。 + +```shell +# 基于用户远程仓库(origin)的 develop 创建 new_branch 分支 +git checkout -b new_branch origin/develop +# 基于上游远程仓库(upstream)的 develop 创建 new_branch 分支 +# 如果需要从 upstream 创建新的分支，需要首先使用 git fetch upstream 获取上游代码 +git checkout -b new_branch upstream/develop +``` + +最终会显示切换到新的分支，输出信息如下 + +``` +Branch new_branch set up to track remote branch develop from upstream. +Switched to a new branch 'new_branch' +``` + + +#### 1.2.4 使用 pre-commit 勾子 + +Paddle 开发人员使用 pre-commit 工具来管理 Git 预提交钩子。 它可以帮助我们格式化源代码(C++，Python)，在提交(commit)前自动检查一些基本事宜（如每个文件只有一个 EOL，Git 中不要添加大文件等）。 + +pre-commit 测试是 Travis-CI 中单元测试的一部分，不满足钩子的 PR 不能被提交到 PaddleClas，首先安装并在当前目录运行它： + +```shell +pip install pre-commit +pre-commit install +``` + +* **注意** + +1. Paddle 使用 clang-format 来调整 C/C++ 源代码格式，请确保 `clang-format` 版本在 3.8 以上。 +2. 通过 `pip install pre-commit` 和 `conda install -c conda-forge pre-commit` 安装的 `yapf` 稍有不同的，PaddleClas 开发人员使用的是 `pip install pre-commit` 。 + + +#### 1.2.5 修改与提交代码 + +可以通过 `git status` 查看改动的文件。 +对 PaddleClas 的 `README.md` 做了一些修改，希望提交上去。则可以通过以下步骤 + +```shell +git add README.md +pre-commit +``` + +重复上述步骤，直到 pre-comit 格式检查不报错。如下所示。 + +![](../../images/quick_start/community/003_precommit_pass.png) + +使用下面的命令完成提交。 + +```shell +git commit -m "your commit info" +``` + + +#### 1.2.6 保持本地仓库最新 + +获取 upstream 的最新代码并更新当前分支。这里的 upstream 来自于 1.2 节的`和远程仓库建立连接`部分。 + +```shell +git fetch upstream +# 如果是希望提交到其他分支，则需要从 upstream 的其他分支 pull 代码，这里是 develop +git pull upstream develop +``` + + +#### 1.2.7 push 到远程仓库 + +```shell +git push origin new_branch +``` + + +#### 1.2.8 提交 Pull Request + +点击 new pull request，选择本地分支和目标分支，如下图所示。在 PR 的描述说明中，填写该 PR 所完成的功能。接下来等待 review，如果有需要修改的地方，参照上述步骤更新 origin 中的对应分支即可。 + +![](../../images/quick_start/community/004_create_pr.png) + +#### 1.2.9 签署 CLA 协议和通过单元测试 + +* 签署 CLA +在首次向 PaddlePaddle 提交 Pull Request 时，您需要您签署一次 CLA(Contributor License Agreement)协议，以保证您的代码可以被合入，具体签署方式如下： + +1. 请您查看 PR 中的 Check 部分，找到 license/cla，并点击右侧 detail，进入 CLA 网站 +2. 点击 CLA 网站中的 `Sign in with GitHub to agree`, 点击完成后将会跳转回您的 Pull Request 页面 + + +#### 1.2.10 删除分支 + +* 删除远程分支 + +在 PR 被 merge 进主仓库后，我们可以在 PR 的页面删除远程仓库的分支。 + +也可以使用 `git push origin :分支名` 删除远程分支，如： + + +```shell +git push origin :new_branch +``` + +* 删除本地分支 + +```shell +# 切换到 develop 分支，否则无法删除当前分支 +git checkout develop + +# 删除 new_branch 分支 +git branch -D new_branch +``` + + +#### 1.2.11 提交代码的一些约定 + +为了使官方维护人员在评审代码时更好地专注于代码本身，请您每次提交代码时，遵守以下约定： + +1）请保证 Travis-CI 中单元测试能顺利通过。如果没过，说明提交的代码存在问题，官方维护人员一般不做评审。 + +2）提交 Pull Request 前： + +请注意 commit 的数量。 + +原因：如果仅仅修改一个文件但提交了十几个 commit，每个 commit 只做了少量的修改，这会给评审人带来很大困扰。评审人需要逐一查看每个 commit 才能知道做了哪些修改，且不排除 commit 之间的修改存在相互覆盖的情况。 + +建议：每次提交时，保持尽量少的 commit，可以通过 `git commit --amend` 补充上次的 commit 。对已经 Push 到远程仓库的多个 commit，可以参考 [squash commits after push](https://stackoverflow.com/questions/5667884/how-to-squash-commits-in-git-after-they-have-been-pushed)。 + +请注意每个 commit 的名称：应能反映当前 commit 的内容，不能太随意。 + +3）如果解决了某个 Issue 的问题，请在该 Pull Request 的第一个评论框中加上： `fix #issue_number`，这样当该 Pull Request 被合并后，会自动关闭对应的 Issue 。关键词包括： close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved，请选择合适的词汇。详细可参考 [Closing issues via commit messages](https://help.github.com/articles/closing-issues-via-commit-messages)。 + +此外，在回复评审人意见时，请您遵守以下约定： + +1）官方维护人员的每一个 review 意见都希望得到回复，这样会更好地提升开源社区的贡献。 + +- 对评审意见同意且按其修改完的，给个简单的 Done 即可； +- 对评审意见不同意的，请给出您自己的反驳理由。 + +2）如果评审意见比较多, + +- 请给出总体的修改情况。 +- 请采用 `start a review` 进行回复，而非直接回复的方式。原因是每个回复都会发送一封邮件，会造成邮件灾难。 + + +## 2. 总结 + +* 开源社区依赖于众多开发者与用户的贡献和反馈，在这里感谢与期待大家向 PaddleClas 提出宝贵的意见与 Pull Request，希望我们可以一起打造一个领先实用全面的图像识别代码仓库！ + + +## 3. 参考文献 +1. [PaddlePaddle 本地开发指南](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/08_contribution/index_cn.html) +2. [向开源框架提交 pr 的过程](https://blog.csdn.net/vim_wj/article/details/78300239) diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/index.rst b/src/PaddleClas/docs/zh_CN/advanced_tutorials/index.rst new file mode 100644 index 0000000..760ec5c --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/index.rst @@ -0,0 +1,13 @@ +高级教程 +================================ + +.. toctree:: + :maxdepth: 2 + + DataAugmentation.md + knowledge_distillation.md + model_prune_quantization.md + code_overview.md + how_to_contribute.md + + diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/knowledge_distillation.md b/src/PaddleClas/docs/zh_CN/advanced_tutorials/knowledge_distillation.md new file mode 100644 index 0000000..d3e6d77 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/knowledge_distillation.md @@ -0,0 +1,275 @@ + +# 知识蒸馏 + +## 目录 + - [1. 模型压缩与知识蒸馏方法简介](#1) + - [2. SSLD 蒸馏策略](#2) + - [2.1 简介](#2.1) + - [2.2 数据选择](#2.2) + - [3. 实验](#3) + - [3.1 教师模型的选择](#3.1) + - [3.2 大数据蒸馏](#3.2) + - [3.3 ImageNet1k 训练集 finetune](#3.3) + - [3.4 数据增广以及基于 Fix 策略的微调](#3.4) + - [3.5 实验过程中的一些问题](#3.5) + - [4. 蒸馏模型的应用](#4) + - [4.1 使用方法](#4.1) + - [4.2 迁移学习 finetune](#4.2) + - [4.3 目标检测](#4.3) + - [5. SSLD 实战](#5) + - [5.1 参数配置](#5.1) + - [5.2 启动命令](#5.2) + - [5.3 注意事项](#5.3) + - [6. 参考文献](#6) + + +## 1. 模型压缩与知识蒸馏方法简介 + +近年来，深度神经网络在计算机视觉、自然语言处理等领域被验证是一种极其有效的解决问题的方法。通过构建合适的神经网络，加以训练，最终网络模型的性能指标基本上都会超过传统算法。 + +在数据量足够大的情况下，通过合理构建网络模型的方式增加其参数量，可以显著改善模型性能，但是这又带来了模型复杂度急剧提升的问题。大模型在实际场景中使用的成本较高。 + +深度神经网络一般有较多的参数冗余，目前有几种主要的方法对模型进行压缩，减小其参数量。如裁剪、量化、知识蒸馏等，其中知识蒸馏是指使用教师模型(teacher model)去指导学生模型(student model)学习特定任务，保证小模型在参数量不变的情况下，得到比较大的性能提升，甚至获得与大模型相似的精度指标 [1]。 PaddleClas 融合已有的蒸馏方法 [2,3]，提供了一种简单的半监督标签知识蒸馏方案(SSLD，Simple Semi-supervised Label Distillation)，基于 ImageNet1k 分类数据集，在 ResNet_vd 以及 MobileNet 系列上的精度均有超过 3% 的绝对精度提升，具体指标如下图所示。 + +![](../../images/distillation/distillation_perform_s.jpg) + +## 2. SSLD 蒸馏策略 + + +### 2.1 简介 + +SSLD 的流程图如下图所示。 + +![](../../images/distillation/ppcls_distillation.png) +首先，我们从 ImageNet22k 中挖掘出了近 400 万张图片，同时与 ImageNet-1k 训练集整合在一起，得到了一个新的包含 500 万张图片的数据集。然后，我们将学生模型与教师模型组合成一个新的网络，该网络分别输出学生模型和教师模型的预测分布，与此同时，固定教师模型整个网络的梯度，而学生模型可以做正常的反向传播。最后，我们将两个模型的 logits 经过 softmax 激活函数转换为 soft label，并将二者的 soft label 做 JS 散度作为损失函数，用于蒸馏模型训练。下面以 MobileNetV3（该模型直接训练，精度为 75.3%）的知识蒸馏为例，介绍该方案的核心关键点（baseline 为 79.12% 的 ResNet50_vd 模型蒸馏 MobileNetV3，训练集为 ImageNet1k 训练集，loss 为 cross entropy loss，迭代轮数为 120epoch，精度指标为 75.6%）。 + +* 教师模型的选择。在进行知识蒸馏时，如果教师模型与学生模型的结构差异太大，蒸馏得到的结果反而不会有太大收益。相同结构下，精度更高的教师模型对结果也有很大影响。相比于 79.12% 的 ResNet50_vd 教师模型，使用 82.4% 的 ResNet50_vd 教师模型可以带来 0.4% 的绝对精度收益(`75.6%->76.0%`)。 + +* 改进 loss 计算方法。分类 loss 计算最常用的方法就是 cross entropy loss，我们经过实验发现，在使用 soft label 进行训练时，相对于 cross entropy loss，KL div loss 对模型性能提升几乎无帮助，但是使用具有对称特性的 JS div loss 时，在多个蒸馏任务上相比 cross entropy loss 均有 0.2% 左右的收益(`76.0%->76.2%`)，SSLD 中也基于 JS div loss 展开实验。 + +* 更多的迭代轮数。蒸馏的 baseline 实验只迭代了 120 个 epoch 。实验发现，迭代轮数越多，蒸馏效果越好，最终我们迭代了 360 epoch，精度指标可以达到 77.1%(`76.2%->77.1%`)。 + +* 无需数据集的真值标签，很容易扩展训练集。 SSLD 的 loss 在计算过程中，仅涉及到教师和学生模型对于相同图片的处理结果（经过 softmax 激活函数处理之后的 soft label），因此即使图片数据不包含真值标签，也可以用来进行训练并提升模型性能。该蒸馏方案的无标签蒸馏策略也大大提升了学生模型的性能上限(`77.1%->78.5%`)。 + +* ImageNet1k 蒸馏 finetune 。 我们仅使用 ImageNet1k 数据，使用蒸馏方法对上述模型进行 finetune，最终仍然可以获得 0.4% 的性能提升(`78.5%->78.9%`)。 + + + +### 2.2 数据选择 + +* SSLD 蒸馏方案的一大特色就是无需使用图像的真值标签，因此可以任意扩展数据集的大小，考虑到计算资源的限制，我们在这里仅基于 ImageNet22k 数据集对蒸馏任务的训练集进行扩充。在 SSLD 蒸馏任务中，我们使用了 `Top-k per class` 的数据采样方案 [3] 。具体步骤如下。 + * 训练集去重。我们首先基于 SIFT 特征相似度匹配的方式对 ImageNet22k 数据集与 ImageNet1k 验证集进行去重，防止添加的 ImageNet22k 训练集中包含 ImageNet1k 验证集图像，最终去除了 4511 张相似图片。部分过滤的相似图片如下所示。 + + ![](../../images/distillation/22k_1k_val_compare_w_sift.png) + + * 大数据集 soft label 获取，对于去重后的 ImageNet22k 数据集，我们使用 `ResNeXt101_32x16d_wsl` 模型进行预测，得到每张图片的 soft label 。 + * Top-k 数据选择，ImageNet1k 数据共有 1000 类，对于每一类，找出属于该类并且得分最高的 `k` 张图片，最终得到一个数据量不超过 `1000*k` 的数据集（某些类上得到的图片数量可能少于 `k` 张）。 + * 将该数据集与 ImageNet1k 的训练集融合组成最终蒸馏模型所使用的数据集，数据量为 500 万。 + + +## 3. 实验 + +* PaddleClas 的蒸馏策略为`大数据集训练 + ImageNet1k 蒸馏 finetune` 的策略。选择合适的教师模型，首先在挑选得到的 500 万数据集上进行训练，然后在 ImageNet1k 训练集上进行 finetune，最终得到蒸馏后的学生模型。 + + +### 3.1 教师模型的选择 + +为了验证教师模型和学生模型的模型大小差异和教师模型的模型精度对蒸馏结果的影响，我们做了几组实验验证。训练策略统一为：`cosine_decay_warmup，lr=1.3, epoch=120, bs=2048`，学生模型均为从头训练。 + +|Teacher Model | Teacher Top1 | Student Model | Student Top1| +|- |:-: |:-: | :-: | +| ResNeXt101_32x16d_wsl | 84.2% | MobileNetV3_large_x1_0 | 75.78% | +| ResNet50_vd | 79.12% | MobileNetV3_large_x1_0 | 75.60% | +| ResNet50_vd | 82.35% | MobileNetV3_large_x1_0 | 76.00% | + + +从表中可以看出 + +> 教师模型结构相同时，其精度越高，最终的蒸馏效果也会更好一些。 +> +> 教师模型与学生模型的模型大小差异不宜过大，否则反而会影响蒸馏结果的精度。 + + +因此最终在蒸馏实验中，对于 ResNet 系列学生模型，我们使用 `ResNeXt101_32x16d_wsl` 作为教师模型；对于 MobileNet 系列学生模型，我们使用蒸馏得到的 `ResNet50_vd` 作为教师模型。 + + +### 3.2 大数据蒸馏 + +基于 PaddleClas 的蒸馏策略为`大数据集训练 + imagenet1k finetune` 的策略。 + +针对从 ImageNet22k 挑选出的 400 万数据，融合 imagenet1k 训练集，组成共 500 万的训练集进行训练，具体地，在不同模型上的训练超参及效果如下。 + + +|Student Model | num_epoch | l2_ecay | batch size/gpu cards | base lr | learning rate decay | top1 acc | +| - |:-: |:-: | :-: |:-: |:-: |:-: | +| MobileNetV1 | 360 | 3e-5 | 4096/8 | 1.6 | cosine_decay_warmup | 77.65% | +| MobileNetV2 | 360 | 1e-5 | 3072/8 | 0.54 | cosine_decay_warmup | 76.34% | +| MobileNetV3_large_x1_0 | 360 | 1e-5 | 5760/24 | 3.65625 | cosine_decay_warmup | 78.54% | +| MobileNetV3_small_x1_0 | 360 | 1e-5 | 5760/24 | 3.65625 | cosine_decay_warmup | 70.11% | +| ResNet50_vd | 360 | 7e-5 | 1024/32 | 0.4 | cosine_decay_warmup | 82.07% | +| ResNet101_vd | 360 | 7e-5 | 1024/32 | 0.4 | cosine_decay_warmup | 83.41% | +| Res2Net200_vd_26w_4s | 360 | 4e-5 | 1024/32 | 0.4 | cosine_decay_warmup | 84.82% | + + +### 3.3 ImageNet1k 训练集 finetune + +对于在大数据集上训练的模型，其学习到的特征可能与 ImageNet1k 数据特征有偏，因此在这里使用 ImageNet1k 数据集对模型进行 finetune。 finetune 的超参和 finetune 的精度收益如下。 + + +|Student Model | num_epoch | l2_ecay | batch size/gpu cards | base lr | learning rate decay | top1 acc | +| - |:-: |:-: | :-: |:-: |:-: |:-: | +| MobileNetV1 | 30 | 3e-5 | 4096/8 | 0.016 | cosine_decay_warmup | 77.89% | +| MobileNetV2 | 30 | 1e-5 | 3072/8 | 0.0054 | cosine_decay_warmup | 76.73% | +| MobileNetV3_large_x1_0 | 30 | 1e-5 | 2048/8 | 0.008 | cosine_decay_warmup | 78.96% | +| MobileNetV3_small_x1_0 | 30 | 1e-5 | 6400/32 | 0.025 | cosine_decay_warmup | 71.28% | +| ResNet50_vd | 60 | 7e-5 | 1024/32 | 0.004 | cosine_decay_warmup | 82.39% | +| ResNet101_vd | 30 | 7e-5 | 1024/32 | 0.004 | cosine_decay_warmup | 83.73% | +| Res2Net200_vd_26w_4s | 360 | 4e-5 | 1024/32 | 0.004 | cosine_decay_warmup | 85.13% | + + +### 3.4 数据增广以及基于 Fix 策略的微调 + +* 基于前文所述的实验结论，我们在训练的过程中加入自动增广(AutoAugment)[4]，同时进一步减小了 l2_decay(4e-5->2e-5)，最终 ResNet50_vd 经过 SSLD 蒸馏策略，在 ImageNet1k 上的精度可以达到 82.99%，相比之前不加数据增广的蒸馏策略再次增加了 0.6% 。 + + +* 对于图像分类任务，在测试的时候，测试尺度为训练尺度的 1.15 倍左右时，往往在不需要重新训练模型的情况下，模型的精度指标就可以进一步提升 [5]，对于 82.99% 的 ResNet50_vd 在 320x320 的尺度下测试，精度可达 83.7%，我们进一步使用 Fix 策略，即在 320x320 的尺度下进行训练，使用与预测时相同的数据预处理方法，同时固定除 FC 层以外的所有参数，最终在 320x320 的预测尺度下，精度可以达到 **84.0%**。 + + +### 3.5 实验过程中的一些问题 + +* 在预测过程中，batch norm 的平均值与方差是通过加载预训练模型得到（设其模式为 test mode）。在训练过程中，batch norm 是通过统计当前 batch 的信息（设其模式为 train mode），与历史保存信息进行滑动平均计算得到，在蒸馏任务中，我们发现通过 train mode，即教师模型的均值与方差实时变化的模式，去指导学生模型，比通过 test mode 蒸馏，得到的学生模型性能更好一些，下面是一组实验结果。因此我们在该蒸馏方案中，均使用 train mode 去得到教师模型的 soft label 。 + +|Teacher Model | Teacher Top1 | Student Model | Student Top1| +|- |:-: |:-: | :-: | +| ResNet50_vd | 82.35% | MobileNetV3_large_x1_0 | 76.00% | +| ResNet50_vd | 82.35% | MobileNetV3_large_x1_0 | 75.84% | + + +## 4. 蒸馏模型的应用 + + +### 4.1 使用方法 + +* 中间层学习率调整。蒸馏得到的模型的中间层特征图更加精细化，因此将蒸馏模型预训练应用到其他任务中时，如果采取和之前相同的学习率，容易破坏中间层特征。而如果降低整体模型训练的学习率，则会带来训练收敛速度慢的问题。因此我们使用了中间层学习率调整的策略。具体地： + * 针对 ResNet50_vd，我们设置一个学习率倍数列表，res block 之前的 3 个 conv2d 卷积参数具有统一的学习率倍数，4 个 res block 的 conv2d 分别有一个学习率参数，共需设置 5 个学习率倍数的超参。在实验中发现。用于迁移学习 finetune 分类模型时，`[0.1,0.1,0.2,0.2,0.3]` 的中间层学习率倍数设置在绝大多数的任务中都性能更好；而在目标检测任务中，`[0.05,0.05,0.05,0.1,0.15]` 的中间层学习率倍数设置能够带来更大的精度收益。 + * 对于 MoblileNetV3_large_x1_0，由于其包含 15 个 block，我们设置每 3 个 block 共享一个学习率倍数参数，因此需要共 5 个学习率倍数的参数，最终发现在分类和检测任务中，`[0.25,0.25,0.5,0.5,0.75]` 的中间层学习率倍数能够带来更大的精度收益。 + + +* 适当的 l2 decay 。不同分类模型在训练的时候一般都会根据模型设置不同的 l2 decay，大模型为了防止过拟合，往往会设置更大的 l2 decay，如 ResNet50 等模型，一般设置为 `1e-4` ；而如 MobileNet 系列模型，在训练时往往都会设置为 `1e-5~4e-5`，防止模型过度欠拟合，在蒸馏时亦是如此。在将蒸馏模型应用到目标检测任务中时，我们发现也需要调节 backbone 甚至特定任务模型模型的 l2 decay，和预训练蒸馏时的 l2 decay 尽可能保持一致。以 Faster RCNN MobiletNetV3 FPN 为例，我们发现仅修改该参数，在 COCO2017 数据集上就可以带来最多 0.5% 左右的精度(mAP)提升（默认 Faster RCNN l2 decay 为 1e-4，我们修改为 1e-5~4e-5 均有 0.3%~0.5% 的提升）。 + + +### 4.2 迁移学习 finetune +* 为验证迁移学习的效果，我们在 10 个小的数据集上验证其效果。在这里为了保证实验的可对比性，我们均使用 ImageNet1k 数据集训练的标准预处理过程，对于蒸馏模型我们也添加了蒸馏模型中间层学习率的搜索。 +* 对于 ResNet50_vd, baseline 为 Top1 Acc 79.12% 的预训练模型基于 grid search 搜索得到的最佳精度，对比实验则为基于该精度对预训练和中间层学习率进一步搜索得到的最佳精度。下面给出 10 个数据集上所有 baseline 和蒸馏模型的精度对比。 + + +| Dataset | Model | Baseline Top1 Acc | Distillation Model Finetune | +|- |:-: |:-: | :-: | +| Oxford102 flowers | ResNete50_vd | 97.18% | 97.41% | +| caltech-101 | ResNete50_vd | 92.57% | 93.21% | +| Oxford-IIIT-Pets | ResNete50_vd | 94.30% | 94.76% | +| DTD | ResNete50_vd | 76.48% | 77.71% | +| fgvc-aircraft-2013b | ResNete50_vd | 88.98% | 90.00% | +| Stanford-Cars | ResNete50_vd | 92.65% | 92.76% | +| SUN397 | ResNete50_vd | 64.02% | 68.36% | +| cifar100 | ResNete50_vd | 86.50% | 87.58% | +| cifar10 | ResNete50_vd | 97.72% | 97.94% | +| Food-101 | ResNete50_vd | 89.58% | 89.99% | + +* 可以看出在上面 10 个数据集上，结合适当的中间层学习率倍数设置，蒸馏模型平均能够带来 1% 以上的精度提升。 + + +### 4.3 目标检测 + +我们基于两阶段目标检测 Faster/Cascade RCNN 模型验证蒸馏得到的预训练模型的效果。 + +* ResNet50_vd + +设置训练与评测的尺度均为 640x640，最终 COCO 上检测指标如下。 + +| Model | train/test scale | pretrain top1 acc | feature map lr | coco mAP | +|- |:-: |:-: | :-: | :-: | +| Faster RCNN R50_vd FPN | 640/640 | 79.12% | [1.0,1.0,1.0,1.0,1.0] | 34.8% | +| Faster RCNN R50_vd FPN | 640/640 | 79.12% | [0.05,0.05,0.1,0.1,0.15] | 34.3% | +| Faster RCNN R50_vd FPN | 640/640 | 82.18% | [0.05,0.05,0.1,0.1,0.15] | 36.3% | + +在这里可以看出，对于未蒸馏模型，过度调整中间层学习率反而降低最终检测模型的性能指标。基于该蒸馏模型，我们也提供了领先的服务端实用目标检测方案，详细的配置与训练代码均已开源，可以参考 [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection/tree/master/configs/rcnn_enhance)。 + + +## 5. SSLD 实战 + +本节将基于 ImageNet-1K 的数据集详细介绍 SSLD 蒸馏实验，如果想快速体验此方法，可以参考 [**30 分钟玩转 PaddleClas（进阶版）**](../quick_start/quick_start_classification_professional.md)中基于 CIFAR100 的 SSLD 蒸馏实验。 + + +### 5.1 参数配置 + +实战部分提供了 SSLD 蒸馏的示例，在 `ppcls/configs/ImageNet/Distillation/mv3_large_x1_0_distill_mv3_small_x1_0.yaml` 中提供了 `MobileNetV3_large_x1_0` 蒸馏 `MobileNetV3_small_x1_0` 的配置文件，用户可以在 `tools/train.sh` 里直接替换配置文件的路径即可使用。 + +```yaml +Arch: + name: "DistillationModel" + # if not null, its lengths should be same as models + pretrained_list: + # if not null, its lengths should be same as models + freeze_params_list: + - True + - False + models: + - Teacher: + name: MobileNetV3_large_x1_0 + pretrained: True + use_ssld: True + - Student: + name: MobileNetV3_small_x1_0 + pretrained: False + + infer_model_name: "Student" +``` + +在参数配置中，`freeze_params_list` 中需要指定模型是否需要冻结参数，`models` 中需要指定 Teacher 模型和 Student 模型，其中 Teacher 模型需要加载预训练模型。用户可以直接在此处更改模型。 + + +### 5.2 启动命令 + +当用户配置完训练环境后，类似于训练其他分类任务，只需要将 `tools/train.sh` 中的配置文件替换成为相应的蒸馏配置文件即可。 + +其中 `train.sh` 中的内容如下： + +```bash + +python -m paddle.distributed.launch \ + --selected_gpus="0,1,2,3" \ + --log_dir=mv3_large_x1_0_distill_mv3_small_x1_0 \ + tools/train.py \ + -c ./ppcls/configs/ImageNet/Distillation/mv3_large_x1_0_distill_mv3_small_x1_0.yaml +``` + +运行 `train.sh` ： + +```bash +sh tools/train.sh +``` + + +### 5.3 注意事项 + +* 用户在使用 SSLD 蒸馏之前，首先需要在目标数据集上训练一个教师模型，该教师模型用于指导学生模型在该数据集上的训练。 + +* 如果学生模型没有加载预训练模型，训练的其他超参数可以参考该学生模型在 ImageNet-1k 上训练的超参数，如果学生模型加载了预训练模型，学习率可以调整到原来的 1/10 或者 1/100 。 + +* 在 SSLD 蒸馏的过程中，学生模型只学习 soft-label 导致训练目标变的更加复杂，建议可以适当的调小 `l2_decay` 的值来获得更高的验证集准确率。 + +* 若用户准备添加无标签的训练数据，只需要将新的训练数据放置在原本训练数据的路径下，生成新的数据 list 即可，另外，新生成的数据 list 需要将无标签的数据添加伪标签（只是为了统一读数据）。 + + +## 6. 参考文献 + +[1] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015. + +[2] Bagherinezhad H, Horton M, Rastegari M, et al. Label refinery: Improving imagenet classification through label progression[J]. arXiv preprint arXiv:1805.02641, 2018. + +[3] Yalniz I Z, Jégou H, Chen K, et al. Billion-scale semi-supervised learning for image classification[J]. arXiv preprint arXiv:1905.00546, 2019. + +[4] Cubuk E D, Zoph B, Mane D, et al. Autoaugment: Learning augmentation strategies from data[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2019: 113-123. + +[5] Touvron H, Vedaldi A, Douze M, et al. Fixing the train-test resolution discrepancy[C]//Advances in Neural Information Processing Systems. 2019: 8250-8260. diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/model_prune_quantization.md b/src/PaddleClas/docs/zh_CN/advanced_tutorials/model_prune_quantization.md new file mode 100644 index 0000000..f6b4afc --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/model_prune_quantization.md @@ -0,0 +1,173 @@ + +# 模型量化、裁剪使用介绍 + +复杂的模型有利于提高模型的性能，但也导致模型中存在一定冗余。此部分提供精简模型的功能，包括两部分：模型量化（量化训练、离线量化）、模型剪枝。 + +其中模型量化将全精度缩减到定点数减少这种冗余，达到减少模型计算复杂度，提高模型推理性能的目的。 +模型量化可以在基本不损失模型的精度的情况下，将 FP32 精度的模型参数转换为 Int8 精度，减小模型参数大小并加速计算，使用量化后的模型在移动端等部署时更具备速度优势。 + +模型剪枝将 CNN 中不重要的卷积核裁剪掉，减少模型参数量，从而降低模型计算复杂度。 + +本教程将介绍如何使用飞桨模型压缩库 PaddleSlim 做 PaddleClas 模型的压缩，即裁剪、量化功能。 +[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)集成了模型剪枝、量化（包括量化训练和离线量化）、蒸馏和神经网络搜索等多种业界常用且领先的模型压缩功能，如果您感兴趣，可以关注并了解。 + +在开始本教程之前，建议先了解 [PaddleClas 模型的训练方法](../models_training/classification.md)以及 [PaddleSlim](https://paddleslim.readthedocs.io/zh_CN/latest/index.html)，相关裁剪、量化方法可以参考[模型裁剪量化算法介绍文档](../algorithm_introduction/model_prune_quantization.md)。 + +----------- + +## 目录 + + - [1. 准备环境](#1) + - [1.1 安装 PaddleSlim](#1.1) + - [1.2 准备训练好的模型](#1.2) + - [2. 快速开始](#2) + - [2.1 模型量化](#2.1) + - [2.1.1 在线量化训练](#2.1.1) + - [2.1.2 离线量化](#2.1.2) + - [2.2 模型剪枝](#2.2) + - [3. 导出模型](#3) + - [4. 模型部署](#4) + - [5. 训练超参数建议](#5) + + + +## 1. 准备环境 +当训练出一个模型后，如果希望进一步的压缩模型大小并加速预测，可使用量化或者剪枝的方法压缩模型。 + +模型压缩主要包括五个步骤： +1. 安装 PaddleSlim +2. 准备训练好的模型 +3. 模型压缩 +4. 导出量化推理模型 +5. 量化模型预测部署 + + +### 1.1 安装 PaddleSlim + +* 可以通过 pip install 的方式进行安装。 + +```bash +pip install paddleslim -i https://pypi.tuna.tsinghua.edu.cn/simple +``` + +* 如果获取 PaddleSlim 的最新特性，可以从源码安装。 + +```bash +git clone https://github.com/PaddlePaddle/PaddleSlim.git +cd Paddleslim +python3.7 setup.py install +``` + + +### 1.2 准备训练好的模型 + +PaddleClas 提供了一系列训练好的[模型](../models/models_intro.md)，如果待量化的模型不在列表中，需要按照[常规训练](../models_training/classification.md)方法得到训练好的模型。 + + +## 2. 快速开始 + +进入 PaddleClas 根目录 + +```bash +cd PaddleClas +``` + +`slim` 训练相关代码已经集成到 `ppcls/engine/` 下，离线量化代码位于 `deploy/slim/quant_post_static.py`。 + + +### 2.1 模型量化 + +量化训练包括离线量化训练和在线量化训练，在线量化训练效果更好，需加载预训练模型，在定义好量化策略后即可对模型进行量化。 + + +#### 2.1.1 在线量化训练 + +训练指令如下： + +* CPU/单卡 GPU + +以 CPU 为例，若使用 GPU，则将命令中改成 `cpu` 改成 `gpu` + +```bash +python3.7 tools/train.py -c ppcls/configs/slim/ResNet50_vd_quantization.yaml -o Global.device=cpu +``` + +其中 `yaml` 文件解析详见[参考文档](../models_training/config_description.md)。为了保证精度，`yaml` 文件中已经使用 `pretrained model`. + + +* 单机多卡/多机多卡启动 + +```bash +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3.7 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ppcls/configs/slim/ResNet50_vd_quantization.yaml +``` + + +#### 2.1.2 离线量化 + +**注意**：目前离线量化，必须使用已经训练好的模型，导出的 `inference model` 进行量化。一般模型导出 `inference model` 可参考[教程](../inference_deployment/export_model.md). + +一般来说，离线量化损失模型精度较多。 + +生成 `inference model` 后，离线量化运行方式如下 + +```bash +python3.7 deploy/slim/quant_post_static.py -c ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml -o Global.save_inference_dir=./deploy/models/class_ResNet50_vd_ImageNet_infer +``` + +`Global.save_inference_dir` 是 `inference model` 存放的目录。 + +执行成功后，在 `Global.save_inference_dir` 的目录下，生成 `quant_post_static_model` 文件夹，其中存储生成的离线量化模型，其可以直接进行预测部署，无需再重新导出模型。 + + +### 2.2 模型剪枝 + +训练指令如下： + +- CPU/单卡 GPU + +以 CPU 为例，若使用 GPU，则将命令中改成 `cpu` 改成 `gpu` + +```bash +python3.7 tools/train.py -c ppcls/configs/slim/ResNet50_vd_prune.yaml -o Global.device=cpu +``` + +- 单机单卡/单机多卡/多机多卡启动 + +```bash +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3.7 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ppcls/configs/slim/ResNet50_vd_prune.yaml +``` + + +## 3. 导出模型 + +在得到在线量化训练、模型剪枝保存的模型后，可以将其导出为 inference model，用于预测部署，以模型剪枝为例： + +```bash +python3.7 tools/export.py \ + -c ppcls/configs/slim/ResNet50_vd_prune.yaml \ + -o Global.pretrained_model=./output/ResNet50_vd/best_model \ + -o Global.save_inference_dir=./inference +``` + + +## 4. 模型部署 + +上述步骤导出的模型可以直接使用 inferecne 进行部署，参考 [inference 部署](../inference_deployment/)。 + +也通过 PaddleLite 的 opt 模型转换工具，完成 inference 模型到移动端模型转换，用于移动端的模型部署。 +移动端模型部署的可参考 [移动端模型部署](../inference_deployment/paddle_lite_deploy.md)。 + + +## 5. 训练超参数建议 + +* 量化、裁剪训练时，建议加载常规训练得到的预训练模型，加速量化训练收敛。 +* 量化训练时，建议初始学习率修改为常规训练的 `1/20~1/10`，同时将训练 epoch 数修改为常规训练的 `1/5~1/2`，学习率策略方面，加上 Warmup，其他配置信息不建议修改。 +* 裁剪训练时，建议超参数配置与普通训练一致。 diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/shitu_deploy.md b/src/PaddleClas/docs/zh_CN/advanced_tutorials/shitu_deploy.md new file mode 100644 index 0000000..064f68e --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/shitu_deploy.md @@ -0,0 +1,97 @@ +# PP-ShiTu迁移应用及调优 + +## 目录 + +- [1.使用官方模型](#1) + - [1.1 下载官方模型及数据准备](#1.1) + - [1.2 建立检索库（gallery）](#1.2) + - [1.3 精度测试](#1.3) +- [2.模型调优](#2) + - [2.1 检测模型调优](#2.1) + - [2.2 识别模型调优](#2.2) +- [3.模型加速](#3) + +在[PP-ShiTu快速开始](../quick_start/quick_start_recognition.md)中，主要展示了`PP-ShiTu`的快速开始demo。那么本文档主要介绍，如何将`PP-ShiTu`应用到自己的需求中，及如何继续调优，优化识别效果。具体可以分成以下三种情况 + +- 直接使用官方模型 +- 根据需求对模型进行调优 +- 模型加速 + +以下分别对上面三种情况进行具体讲解。 + + + +## 1. 使用官方模型 + +对于其他的应用，首先建议使用此种方法，因为PaddleClas官方模型，是经过大量数据训练得到的，具有强大泛化性能的轻量级通用模型。其速度较快、精度较高，在Intel cpu上直接部署，也能快速得到结果。同时，直接使用官方模型，则无需训练新的模型，节约时间，方便快速部署。 + +使用官方模型具体步骤如下： + +- 下载模型及数据准备 +- 检索库更新 +- 精度测试 + + + +### 1.1 下载官方模型及数据准备 + +模型下载及pipline 运行详见[图像识别快速开始](../quick_start/quick_start_recognition.md) + +下载模型后，要准备相应的数据，即所迁移应用的具体数据，数据量根据实际情况，自行决定，但是不能太少，会影响精度。将准备的数据分成两部分：1）建库图像（gallery），2）测试图像。其中建库数据无需过多，但需保证每个类别包含此类别物体不同角度的图像，建议每个类别至少5张图，请根据实际情况，具体调节。 + +数据标注工具可以使用[lebalme](https://github.com/wkentaro/labelme)。标注数据时。请标注待识别物体的的包围框（BoundingBox），注意只需要标注**建库图像**。。 + +建议一个类别一共准备30张图左右，其中约至少5张图作为建库图像，剩下的作为测试图像。 + + + +### 1.2 建立检索库（gallery） + +对于加入检索的数据，每个类别尽量准备此类别的各角度的图像，丰富类别信息。准备的图像只能包含此类别，同时图像背景尽可能的少、简单。即将要加入检索根据标注的包围框信息，裁剪出bbox图像作为新的要加入的图像，以提高检索库的图像质量。 + +收集好图像后，数据整理及建库流程详见[图像识别快速开始](../quick_start/quick_start_recognition.md)中`3.2 建立新的索引库` + + + +### 1.3 精度测试 + +使用测试图像，对整个pipline进行简单的精度测试。如发现类别不正确，则需对gallery进行调整，将不正确的测试图像的相似图片（标注并裁剪出没有背景的物体）加入gallery中，反复迭代。经过调整后，可以测试出整个pipeline的精度。如果精度能够满足需求，则可继续使用。若精度不达预期，则需对模型进行调优，参考下面文档。 + + + +## 2. 模型调优 + +在使用官方模型之后，如果发现精度不达预期，则可对模型进行训练调优。同时，根据官方模型的结果，需要进一步大概判断出 检测模型精度、还是识别模型精度问题。不同模型的调优，可参考以下文档。 + + + +### 2.1 检测模型调优 + +`PP-ShiTu`中检测模型采用的 `PicoDet `算法，具体算法请参考[此文档](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/picodet)。检测模型的训练及调优，请参考[此文档](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.3/README_cn.md)。 + +对模型进行训练的话，需要自行准备数据，并对数据进行标注，建议一个类别至少准备200张标注图像，并将标注图像及groudtruth文件转成coco文件格式，以方便使用PaddleDetection进行训练。主体检测的预训练权重及相关配置文件相见[主体检测文档](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/application/mainbody_detection)。训练的时候，请加载主体检测的预训练权重。 + + + +### 2.2 识别模型调优 + +在使用官方模型后，如果不满足精度需求，则可以参考此部分文档，进行模型调优 + +因为要对模型进行训练，所以收集自己的数据集。数据准备及相应格式请参考：[特征提取文档](../image_recognition_pipeline/feature_extraction.md)中 `4.1数据准备`部分、[识别数据集说明](../data_preparation/recognition_dataset.md)。值得注意的是，此部分需要准备大量的数据，以保证识别模型效果。训练配置文件参考：[通用识别模型配置文件](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml)，训练方法参考：[识别模型训练](../models_training/recognition.md) + +- 数据增强：根据实际情况选择不同数据增强方法。如：实际应用中数据遮挡比较严重，建议添加`RandomErasing`增强方法。详见[数据增强文档](./DataAugmentation.md) +- 换不同的`backbone`，一般来说，越大的模型，特征提取能力更强。不同`backbone`详见[模型介绍](../models/models_intro.md) +- 选择不同的`Metric Learning`方法。不同的`Metric Learning`方法，对不同的数据集效果可能不太一样，建议尝试其他`Loss`,详见[Metric Learning](../algorithm_introduction/metric_learning.md) +- 采用蒸馏方法，对小模型进行模型能力提升，详见[模型蒸馏](../algorithm_introduction/knowledge_distillation.md) +- 增补数据集。针对错误样本，添加badcase数据 + +模型训练完成后，参照[1.2 检索库更新](#1.2)进行检索库更新。同时，对整个pipeline进行测试，如果精度不达预期，则重复此步骤。 + + + +## 3. 模型加速 + +模型加速主要以下几种方法： + +- 替换小模型：一般来说，越小的模型预测速度相对越快 +- 模型裁剪、量化：请参考文档[模型压缩](./model_prune_quantization.md)，压缩配置文件修改请参考[slim相关配置文件](../../../ppcls/configs/slim/)。 diff --git a/src/PaddleClas/docs/zh_CN/advanced_tutorials/theseus_layer.md b/src/PaddleClas/docs/zh_CN/advanced_tutorials/theseus_layer.md new file mode 100644 index 0000000..56f2c97 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/advanced_tutorials/theseus_layer.md @@ -0,0 +1,275 @@ +# TheseusLayer 使用说明 + +基于 TheseusLayer 构建的网络模型，支持网络截断、返回网络中间层输出和修改网络中间层的功能。 + +--- + +## 目录 + +- [1. 前言](#1) +- [2. 网络层描述符说明](#2) +- [3. 功能介绍](#3) + - [3.1 网络截断（stop_after）](#3.1) + - [3.2 返回网络中间层输出（update_res）](#3.2) + - [3.3 修改网络中间层（upgrade_sublayer）](#3.3) + + + +## 1. 前言 + +`TheseusLayer` 是继承了 `nn.Layer` 的子类，使用 `TheseusLayer` 作为父类构建的网络模型，可以通过 `TheseusLayer` 的 `stop_after()`、`update_res()` 和 `upgrade_sublayer()` 实现网络截断、返回中间层输出以及修改网络中间层的功能。目前 PaddleClas 中 `ppcls.arch.backbone.legendary_models` 下的所有模型均支持上述操作。 + +如需基于 `TheseusLayer` 构建新的网络结构，只需继承 `TheseusLayer` 即可： + +```python +from ppcls.arch.backbone.base.theseus_layer import TheseusLayer + +class net(TheseusLayer): + def __init__(): + super().__init__() + + def forward(x): + pass +``` + + + +## 2. 网络层描述符说明 + +使用 `TheseusLayer` 提供的方法对模型进行操作/修改时，需要通过参数指定网络中间层，因此 `TheseusLayer` 规定了用于描述网络中间层的网络层描述符。 + +网络层描述符的使用需要符合以下规则： +* 为 Python 字符串（str）类型； +* 使用网络层对象的变量名指定该网络层； +* 以 `.` 作为网络层级的分隔符； +* 对于 `nn.Sequential` 类型或是 `nn.LayerList` 类型的层，使用 `["index"]` 指定其子层。 + +以 `MobileNetV1` 网络为例，其模型结构定义在 [MobileNetV1](../../../ppcls/arch/backbone/legendary_models/mobilenet_v1.py)，为方便说明，可参考下方网络结构及不同网络层所对应的网络层描述符。可以清晰看出，对于 `MobileNetV1` 网络的任一子层，均可按层级结构逐层指定，不同层级结构间使用 `.` 进行分隔即可。 + +```shell +# 网络层对象的变量名（该对象所属类）....................(该网络层对应的网络层描述符) + +MobileNetV1 +├── conv (ConvBNLayer)............................("conv") +│   ├── conv (nn.Conv2D)..........................("conv.conv") +│   ├── bn (nn.BatchNorm).........................("conv.bn") +│   └── relu (nn.ReLU)............................("conv.relu") +│ +├── blocks (nn.Sequential)........................("blocks") +│   ├── blocks0 (DepthwiseSeparable)..............("blocks[0]") +│   │   ├── depthwise_conv (ConvBNLayer)..........("blocks[0].depthwise_conv") +│   │   │   ├── conv (nn.Conv2D)..................("blocks[0].depthwise_conv.conv") +│   │   │   ├── bn (nn.BatchNorm).................("blocks[0].depthwise_conv.bn") +│   │   │   └── relu (nn.ReLU)....................("blocks[0].depthwise_conv.relu") +│   │   └── pointwise_conv (ConvBNLayer)..........("blocks[0].pointwise_conv") +│   │   ├── conv (nn.Conv2D)..................("blocks[0].pointwise_conv.conv") +│   │   ├── bn (nn.BatchNorm).................("blocks[0].pointwise_conv.bn") +│   │   └── relu (nn.ReLU)....................("blocks[0].pointwise_conv.relu") +│   . +│   . +│   . +│   └── blocks12 (DepthwiseSeparable).............("blocks[12]") +│      ├── depthwise_conv (ConvBNLayer)..........("blocks[0].depthwise_conv") +│      │   ├── conv (nn.Conv2D)..................("blocks[0].depthwise_conv.conv") +│      │   ├── bn (nn.BatchNorm).................("blocks[0].depthwise_conv.bn") +│      │   └── relu (nn.ReLU)....................("blocks[0].depthwise_conv.relu") +│      └── pointwise_conv (ConvBNLayer)..........("blocks[0].pointwise_conv") +│      ├── conv (nn.Conv2D)..................("blocks[0].pointwise_conv.conv") +│      ├── bn (nn.BatchNorm).................("blocks[0].pointwise_conv.bn") +│      └── relu (nn.ReLU)....................("blocks[0].pointwise_conv.relu") +│ +├── avg_pool (nn.AdaptiveAvgPool2D)...............("avg_pool") +│ +├── flatten (nn.Flatten)..........................("flatten") +│ +└── fc (nn.Linear)................................("fc") +``` + +因此，对于 `MobileNetV1` 网络： +* 网络层描述符 `flatten`，其指定了网络 `MobileNetV1` 的 `flatten` 这一层。 +* 网络层描述符 `blocks[5]`，其指定了网络 `MobileNetV1` 的 `blocks` 层中的第 `6` 个 `DepthwiseSeparable` 对象这一层； +* 网络层描述符 `blocks[0].depthwise_conv.conv`，其指定了网络 `MobileNetV1` 的 `blocks` 层中的第 `1` 个 `DepthwiseSeparable` 对象中的 `depthwise_conv` 中的 `conv` 这一层； + + + +## 3. 方法说明 + +PaddleClas 提供的 backbone 网络均基于图像分类数据集训练得到，因此网络的尾部带有用于分类的全连接层，而在特定任务场景下，需要去掉分类的全连接层。在部分下游任务中，例如目标检测场景，需要获取到网络中间层的输出结果，也可能需要对网络的中间层进行修改，因此 `TheseusLayer` 提供了 3 个接口函数用于实现不同的修改功能。 + + + +### 3.1 网络截断（stop_after） + +```python +def stop_after(self, stop_layer_name: str) -> bool: + """stop forward and backward after 'stop_layer_name'. + + Args: + stop_layer_name (str): The name of layer that stop forward and backward after this layer. + + Returns: + bool: 'True' if successful, 'False' otherwise. + """ +``` + +该方法可通过参数 `stop_layer_name` 指定网络中的特定子层，并停止该层之后的所有层的前后向传输，在逻辑上，该层之后不再有其他网络层。 + +* 参数： + * `stop_layer_name`： `str` 类型的对象，用于指定网络子层的网络层描述符。关于网络层描述符的具体规则，请查看[网络层描述符说明](#2)。 +* 返回值： + * 当该方法成功执行时，其返回值为 `True`，否则为 `False`。 + +以 `MobileNetV1` 网络为例，参数 `stop_layer_name` 为 `"blocks[0].depthwise_conv.conv"`，具体效果可以参考下方代码案例进行尝试。 + +```python +# cd or pip install paddleclas to import paddleclas +import paddleclas + +net = paddleclas.MobileNetV1() +print("========== the origin mobilenetv1 net arch ==========") +print(net) + +res = net.stop_after(stop_layer_name="blocks[0].depthwise_conv.conv") +print("The result returned by stop_after(): ", res) +# The result returned by stop_after(): True + +print("\n\n========== the truncated mobilenetv1 net arch ==========") +print(net) +``` + + + +### 3.2 返回网络中间层输出（update_res） + +```python +def update_res( + self, + return_patterns: Union[str, List[str]]) -> Dict[str, nn.Layer]: + """update the result(s) to be returned. + + Args: + return_patterns (Union[str, List[str]]): The name of layer to return output. + + Returns: + Dict[str, nn.Layer]: The pattern(str) and corresponding layer(nn.Layer) that have been set successfully. + """ +``` + +该方法可通过参数 `return_patterns` 指定一层（str 对象）或多层（list 对象）网络的中间子层，并在网络前向时，将指定层的输出结果与网络的最终结果一同返回。 + +* 参数： + * `return_patterns`：作为网络层描述符的 `str` 对象，或是 `str` 对象所组成的 `list` 对象，其元素为用于指定网络子层的网络层描述符。关于网络层描述符的具体规则，请查看[网络层描述符说明](#2)。 +* 返回值： + * 该方法的返回值为 `list` 对象，元素为设置成功的子层的网络层描述符。 + +以 `MobileNetV1` 网络为例，当 `return_patterns` 为 `["blocks[0]", "blocks[2]", "blocks[4]", "blocks[10]"]`，在网络前向推理时，网络的输出结果将包含以上 4 层的输出和网络最终的输出，具体效果可以参考下方代码案例进行尝试。 + +```python +import numpy as np +import paddle + +# cd or pip install paddleclas to import paddleclas +import paddleclas + +np_input = np.zeros((1, 3, 224, 224)) +pd_input = paddle.to_tensor(np_input, dtype="float32") + +net = paddleclas.MobileNetV1(pretrained=True) + +output = net(pd_input) +print("The output's type of origin net: ", type(output)) +# The output's type of origin net: + +res = net.update_res(return_patterns=["blocks[0]", "blocks[2]", "blocks[4]", "blocks[10]"]) +print("The result returned by update_res(): ", res) +# The result returned by update_res(): ['blocks[0]', 'blocks[2]', 'blocks[4]', 'blocks[10]'] + +output = net(pd_input) +print("The output's keys of processed net: ", output.keys()) +# The output's keys of net: dict_keys(['output', 'blocks[0]', 'blocks[2]', 'blocks[4]', 'blocks[10]']) +# 网络前向输出 output 为 dict 类型对象，其中，output["output"] 为网络最终输出，output["blocks[0]"] 等为网络中间层输出结果 +``` + +除了通过调用方法 `update_res()` 的方式之外，也同样可以在实例化网络对象时，通过指定参数 `return_patterns` 实现相同效果： + +```python +net = paddleclas.MobileNetV1(pretrained=True, return_patterns=["blocks[0]", "blocks[2]", "blocks[4]", "blocks[10]"]) +``` + +并且在实例化网络对象时，还可以通过参数 `return_stages` 指定网络不同 `stage` 的输出，如下方代码所示： + +```python +# 当 `return_stages` 为 `True`，会将网络所有 stage 的前向输出一并返回，如下所示： +net = paddleclas.MobileNetV1(pretrained=True, return_stages=True) + +# 当 `return_stages` 为 list 对象，可以指定需要返回输出结果的 stage 的序号，如下所示： +net = paddleclas.MobileNetV1(pretrained=True, return_stages=[0, 1, 2, 3]) +``` + + + +### 3.3 修改网络中间层（upgrade_sublayer） + +```python +def upgrade_sublayer(self, + layer_name_pattern: Union[str, List[str]], + handle_func: Callable[[nn.Layer, str], nn.Layer] + ) -> Dict[str, nn.Layer]: + """use 'handle_func' to modify the sub-layer(s) specified by 'layer_name_pattern'. + + Args: + layer_name_pattern (Union[str, List[str]]): The name of layer to be modified by 'handle_func'. + handle_func (Callable[[nn.Layer, str], nn.Layer]): The function to modify target layer specified by 'layer_name_pattern'. The formal params are the layer(nn.Layer) and pattern(str) that is (a member of) layer_name_pattern (when layer_name_pattern is List type). And the return is the layer processed. + + Returns: + Dict[str, nn.Layer]: The key is the pattern and corresponding value is the result returned by 'handle_func()'. + """ +``` + +该方法可通过参数 `layer_name_pattern` 指定一层（str 对象）或多层（list 对象）网络中间子层，并使用参数 `handle_func` 所指定的函数对指定的子层进行修改。 + +* 参数： + * `layer_name_pattern`：作为网络层描述符的 `str` 对象，或是 `str` 对象所组成的 `list` 对象，其元素为用于指定网络子层的网络层描述符。关于网络层描述符的具体规则，请查看[网络层描述符说明](#2)。 + * `handle_func`：有 2 个形参的可调用对象，第 1 个形参为 `nn.Layer` 类型，第 2 个形参为 `str` 类型，该可调用对象返回值必须为 `nn.Layer` 类型对象或是有 `forward` 方法的对象。 +* 返回值： + * 该方法的返回值为 `list` 对象，元素为修改成功的网络子层的网络层描述符。 + +`upgrade_sublayer` 方法会根据 `layer_name_pattern` 查找对应的网络子层，并将查找到的子层和其 `pattern` 传入可调用对象 `handle_func`，并使用 `handle_func` 的返回值替换该层。 + +以 `MobileNetV1` 网络为例，将网络最后的 2 个 block 中的深度可分离卷积（depthwise_conv）改为 `5*5` 大小的卷积核，同时将 padding 改为 `2`，如下方代码所示： + +```python +from paddle import nn + +# cd or pip install paddleclas to import paddleclas +import paddleclas + +# 该函数必须有两个形参 +# 第一个形参用于接受指定的网络中间子层 +# 第二个形参用于接受指定网络子层的网络层描述符 +def rep_func(layer: nn.Layer, pattern: str): + new_layer = nn.Conv2D( + # layer 为 blocks[11].depthwise_conv.conv 或 + # blocks[12].depthwise_conv.conv 所对应的网络中间子层 + # 因此，新的网络层（new_layer）与被替换掉的网络层具有相同的 + # in_channels 属性和 out_channels 属性 + in_channels=layer._in_channels, + out_channels=layer._out_channels, + kernel_size=5, + padding=2 + ) + # 该函数返回值为新的网络层 + # upgrade_sublayer() 方法将使用该返回值替换对应的网络中间子层 + return new_layer + +net = paddleclas.MobileNetV1(pretrained=True) +print("========== the origin mobilenetv1 net arch ==========") +print(net) + +res = net.upgrade_sublayer(layer_name_pattern=["blocks[11].depthwise_conv.conv", "blocks[12].depthwise_conv.conv"], handle_func=rep_func) +print("The result returned by upgrade_sublayer() is", res) +# The result returned by upgrade_sublayer() is ['blocks[11].depthwise_conv.conv', 'blocks[12].depthwise_conv.conv'] + +print("\n\n========== the upgraded mobilenetv1 net arch ==========") +print(net) +``` diff --git a/src/PaddleClas/docs/zh_CN/algorithm_introduction/DataAugmentation.md b/src/PaddleClas/docs/zh_CN/algorithm_introduction/DataAugmentation.md new file mode 100644 index 0000000..32bdfd9 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/algorithm_introduction/DataAugmentation.md @@ -0,0 +1,277 @@ +# 数据增强 +------ +## 目录 + + - [1. 数据增强简介](#1) + - [2. 常用数据增强方法](#2) + - [2.1 图像变换类](#2.1) + - [2.1.1 AutoAugment](#2.1.1) + - [2.1.2 RandAugment](#2.1.2) + - [2.1.3 TimmAutoAugment](#2.1.3) + - [2.2 图像裁剪类](#2.2) + - [2.2.1 Cutout](#2.2.1) + - [2.2.2 RandomErasing](#2.2.2) + - [2.2.3 HideAndSeek](#2.2.3) + - [2.2.4 GridMask](#2.2.4) + - [2.3 图像混叠类](#2.3) + - [2.3.1 Mixup](#2.3.1) + - [2.3.2 Cutmix](#2.3.2) + + +## 1. 数据增强简介 + +在图像分类任务中，图像数据的增广是一种常用的正则化方法，常用于数据量不足或者模型参数较多的场景。在本章节中，我们将对除 ImageNet 分类任务标准数据增强外的 8 种数据增强方式进行简单的介绍和对比，用户也可以将这些增广方法应用到自己的任务中，以获得模型精度的提升。这 8 种数据增强方式在 ImageNet 上的精度指标如下所示。 + +![](../../images/image_aug/main_image_aug.png) + + +## 2. 常用数据增强方法 + +如果没有特殊说明，本章节中所有示例为 ImageNet 分类，并且假设最终输入网络的数据维度为：`[batch-size, 3, 224, 224]` + +其中 ImageNet 分类训练阶段的标准数据增强方式分为以下几个步骤： + +1. 图像解码：简写为 `ImageDecode` +2. 随机裁剪到长宽均为 224 的图像：简写为 `RandCrop` +3. 水平方向随机翻转：简写为 `RandFlip` +4. 图像数据的归一化：简写为 `Normalize` +5. 图像数据的重排，`[224, 224, 3]` 变为 `[3, 224, 224]`：简写为 `Transpose` +6. 多幅图像数据组成 batch 数据，如 `batch-size` 个 `[3, 224, 224]` 的图像数据拼组成 `[batch-size, 3, 224, 224]`：简写为 `Batch` + +相比于上述标准的图像增广方法，研究者也提出了很多改进的图像增广策略，这些策略均是在标准增广方法的不同阶段插入一定的操作，基于这些策略操作所处的不同阶段，我们将其分为了三类： + +1. 对 `RandCrop` 后的 224 的图像进行一些变换: AutoAugment，RandAugment +2. 对 `Transpose` 后的 224 的图像进行一些裁剪: CutOut，RandErasing，HideAndSeek，GridMask +3. 对 `Batch` 后的数据进行混合: Mixup，Cutmix + +增广后的可视化效果如下所示。 + +![](../../images/image_aug/image_aug_samples_s.jpg) + +具体如下表所示： + + +| 变换方法 | 输入 | 输出 | Auto-
Augment\[1\] | Rand-
Augment\[2\] | CutOut\[3\] | Rand
Erasing\[4\] | HideAnd-
Seek\[5\] | GridMask\[6\] | Mixup\[7\] | Cutmix\[8\] | +|-------------|---------------------------|---------------------------|------------------|------------------|-------------|------------------|------------------|---------------|------------|------------| +| Image
Decode | Binary | (224, 224, 3)
uint8 | Y | Y | Y | Y | Y | Y | Y | Y | +| RandCrop | (:, :, 3)
uint8 | (224, 224, 3)
uint8 | Y | Y | Y | Y | Y | Y | Y | Y | +| **Process** | (224, 224, 3)
uint8 | (224, 224, 3)
uint8 | Y | Y | \- | \- | \- | \- | \- | \- | +| RandFlip | (224, 224, 3)
uint8 | (224, 224, 3)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| Normalize | (224, 224, 3)
uint8 | (3, 224, 224)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| Transpose | (224, 224, 3)
float32 | (3, 224, 224)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| **Process** | (3, 224, 224)
float32 | (3, 224, 224)
float32 | \- | \- | Y | Y | Y | Y | \- | \- | +| Batch | (3, 224, 224)
float32 | (N, 3, 224, 224)
float32 | Y | Y | Y | Y | Y | Y | Y | Y | +| **Process** | (N, 3, 224, 224)
float32 | (N, 3, 224, 224)
float32 | \- | \- | \- | \- | \- | \- | Y | Y | + + +PaddleClas 中集成了上述所有的数据增强策略，每种数据增强策略的参考论文与参考开源代码均在下面的介绍中列出。下文将介绍这些策略的原理与使用方法，并以下图为例，对变换后的效果进行可视化。为了说明问题，本章节中将 `RandCrop` 替换为 `Resize`。 + +![][test_baseline] + +### 2.1 图像变换类 + +图像变换类指的是对 `RandCrop` 后的 224 的图像进行一些变换，主要包括 + ++ AutoAugment ++ RandAugment ++ TimmAutoAugment + +#### 2.1.1 AutoAugment + +论文地址：[https://arxiv.org/abs/1805.09501v1](https://arxiv.org/abs/1805.09501v1) + +开源代码 github 地址：[https://github.com/DeepVoltaire/AutoAugment](https://github.com/DeepVoltaire/AutoAugment) + +不同于常规的人工设计图像增广方式，AutoAugment 是在一系列图像增广子策略的搜索空间中通过搜索算法找到的适合特定数据集的图像增广方案。针对 ImageNet 数据集，最终搜索出来的数据增强方案包含 25 个子策略组合，每个子策略中都包含两种变换，针对每幅图像都随机的挑选一个子策略组合，然后以一定的概率来决定是否执行子策略中的每种变换。 + +经过 AutoAugment 数据增强后结果如下图所示。 + +![][test_autoaugment] + +#### 2.1.2 RandAugment + +论文地址：[https://arxiv.org/pdf/1909.13719.pdf](https://arxiv.org/pdf/1909.13719.pdf) + +开源代码 github 地址：[https://github.com/heartInsert/randaugment](https://github.com/heartInsert/randaugment) + + +`AutoAugment` 的搜索方法比较暴力，直接在数据集上搜索针对该数据集的最优策略，其计算量很大。在 `RandAugment` 文章中作者发现，一方面，针对越大的模型，越大的数据集，使用 `AutoAugment` 方式搜索到的增广方式产生的收益也就越小；另一方面，这种搜索出的最优策略是针对该数据集的，其迁移能力较差，并不太适合迁移到其他数据集上。 + +在 `RandAugment` 中，作者提出了一种随机增广的方式，不再像 `AutoAugment` 中那样使用特定的概率确定是否使用某种子策略，而是所有的子策略都会以同样的概率被选择到，论文中的实验也表明这种数据增强方式即使在大模型的训练中也具有很好的效果。 + + +经过 RandAugment 数据增强后结果如下图所示。 + +![][test_randaugment] + +#### 2.1.3 TimmAutoAugment + +开源代码 github 地址：[https://github.com/rwightman/pytorch-image-models/blob/master/timm/data/auto_augment.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/data/auto_augment.py) + +`TimmAutoAugment` 是开源作者对 AutoAugment 和 RandAugment 的改进，事实证明，其在很多视觉任务上有更好的表现，目前绝大多数 VisionTransformer 模型都是基于 TimmAutoAugment 去实现的。 + + +### 2.2 图像裁剪类 + +图像裁剪类主要是对 `Transpose` 后的 224 的图像进行一些裁剪，并将裁剪区域的像素值置为特定的常数（默认为 0），主要包括： + ++ CutOut ++ RandErasing ++ HideAndSeek ++ GridMask + +图像裁剪的这些增广并非一定要放在归一化之后，也有不少实现是放在归一化之前的，也就是直接对 uint8 的图像进行操作，两种方式的差别是：如果直接对 uint8 的图像进行操作，那么再经过归一化之后被裁剪的区域将不再是纯黑或纯白（减均值除方差之后像素值不为 0）。而对归一后之后的数据进行操作，裁剪的区域会是纯黑或纯白。 + +上述的裁剪变换思路是相同的，都是为了解决训练出的模型在有遮挡数据上泛化能力较差的问题，不同的是他们的裁剪方式、区域不太一样。 + +#### 2.2.1 Cutout + +论文地址：[https://arxiv.org/abs/1708.04552](https://arxiv.org/abs/1708.04552) + +开源代码 github 地址：[https://github.com/uoguelph-mlrg/Cutout](https://github.com/uoguelph-mlrg/Cutout) + +Cutout 可以理解为 Dropout 的一种扩展操作，不同的是 Dropout 是对图像经过网络后生成的特征进行遮挡，而 Cutout 是直接对输入的图像进行遮挡，相对于 Dropout 对噪声的鲁棒性更好。作者在论文中也进行了说明，这样做法有以下两点优势：(1)通过 Cutout 可以模拟真实场景中主体被部分遮挡时的分类场景；(2)可以促进模型充分利用图像中更多的内容来进行分类，防止网络只关注显著性的图像区域，从而发生过拟合。 + + +经过 RandAugment 数据增强后结果如下图所示。 + +![][test_cutout] + +#### 2.2.2 RandomErasing + +论文地址：[https://arxiv.org/pdf/1708.04896.pdf](https://arxiv.org/pdf/1708.04896.pdf) + +开源代码 github 地址：[https://github.com/zhunzhong07/Random-Erasing](https://github.com/zhunzhong07/Random-Erasing) + +`RandomErasing` 与 `Cutout` 方法类似，同样是为了解决训练出的模型在有遮挡数据上泛化能力较差的问题，作者在论文中也指出，随机裁剪的方式与随机水平翻转具有一定的互补性。作者也在行人再识别(REID)上验证了该方法的有效性。与 `Cutout` 不同的是，在 `RandomErasing` 中，图片以一定的概率接受该种预处理方法，生成掩码的尺寸大小与长宽比也是根据预设的超参数随机生成。 + + +PaddleClas 中 `RandomErasing` 的使用方法如下所示。 + +经过 RandomErasing 数据增强后结果如下图所示。 + +![][test_randomerassing] + + +#### 2.2.3 HideAndSeek + +论文地址：[https://arxiv.org/pdf/1811.02545.pdf](https://arxiv.org/pdf/1811.02545.pdf) + +开源代码 github 地址：[https://github.com/kkanshul/Hide-and-Seek](https://github.com/kkanshul/Hide-and-Seek) + + +`HideAndSeek` 论文将图像分为若干块区域(patch)，对于每块区域，都以一定的概率生成掩码，不同区域的掩码含义如下图所示。 + + +![][hide_and_seek_mask_expanation] + + +PaddleClas 中 `HideAndSeek` 的使用方法如下所示。 + + +经过 HideAndSeek 数据增强后结果如下图所示。 + +![][test_hideandseek] + + +#### 2.2.4 GridMask +论文地址：[https://arxiv.org/abs/2001.04086](https://arxiv.org/abs/2001.04086) + +开源代码 github 地址：[https://github.com/akuxcw/GridMask](https://github.com/akuxcw/GridMask) + + +作者在论文中指出，此前存在的基于对图像 crop 的方法存在两个问题，如下图所示： + +1. 过度删除区域可能造成目标主体大部分甚至全部被删除，或者导致上下文信息的丢失，导致增广后的数据成为噪声数据； +2. 保留过多的区域，对目标主体及上下文基本产生不了什么影响，失去增广的意义。 + +![][gridmask-0] + +因此如果避免过度删除或过度保留成为需要解决的核心问题。 + + +`GridMask` 是通过生成一个与原图分辨率相同的掩码，并将掩码进行随机翻转，与原图相乘，从而得到增广后的图像，通过超参数控制生成的掩码网格的大小。 + + +在训练过程中，有两种以下使用方法： +1. 设置一个概率 p，从训练开始就对图片以概率 p 使用 `GridMask` 进行增广。 +2. 一开始设置增广概率为 0，随着迭代轮数增加，对训练图片进行 `GridMask` 增广的概率逐渐增大，最后变为 p。 + +论文中验证上述第二种方法的训练效果更好一些。 + +经过 GridMask 数据增强后结果如下图所示。 + +![][test_gridmask] + + +### 2.3 图像混叠类 + +图像混叠主要对 `Batch` 后的数据进行混合，包括： + ++ Mixup ++ Cutmix + +前文所述的图像变换与图像裁剪都是针对单幅图像进行的操作，而图像混叠是对两幅图像进行融合，生成一幅图像，两种方法的主要区别为混叠的方式不太一样。 + +#### 2.3.1 Mixup + +论文地址：[https://arxiv.org/pdf/1710.09412.pdf](https://arxiv.org/pdf/1710.09412.pdf) + +开源代码 github 地址：[https://github.com/facebookresearch/mixup-cifar10](https://github.com/facebookresearch/mixup-cifar10) + +Mixup 是最先提出的图像混叠增广方案，其原理简单、方便实现，不仅在图像分类上，在目标检测上也取得了不错的效果。为了便于实现，通常只对一个 batch 内的数据进行混叠，在 `Cutmix` 中也是如此。 + +如下是 `imaug` 中的实现，需要指出的是，下述实现会出现对同一幅进行相加的情况，也就是最终得到的图和原图一样，随着 `batch-size` 的增加这种情况出现的概率也会逐渐减小。 + + +经过 Mixup 数据增强结果如下图所示。 + +![][test_mixup] + +#### 2.3.2 Cutmix + +论文地址：[https://arxiv.org/pdf/1905.04899v2.pdf](https://arxiv.org/pdf/1905.04899v2.pdf) + +开源代码 github 地址：[https://github.com/clovaai/CutMix-PyTorch](https://github.com/clovaai/CutMix-PyTorch) + +与 `Mixup` 直接对两幅图进行相加不一样，`Cutmix` 是从一幅图中随机裁剪出一个 `ROI`，然后覆盖当前图像中对应的区域，代码实现如下所示： + +经过 Cutmix 数据增强后结果如下图所示。 + +![][test_cutmix] + +关于数据增强相关的实战部分实参考[数据增强实战](../advanced_tutorials/DataAugmentation.md)。 + + +## 参考文献 + +[1] Cubuk E D, Zoph B, Mane D, et al. Autoaugment: Learning augmentation strategies from data[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2019: 113-123. + + +[2] Cubuk E D, Zoph B, Shlens J, et al. Randaugment: Practical automated data augmentation with a reduced search space[J]. arXiv preprint arXiv:1909.13719, 2019. + +[3] DeVries T, Taylor G W. Improved regularization of convolutional neural networks with cutout[J]. arXiv preprint arXiv:1708.04552, 2017. + +[4] Zhong Z, Zheng L, Kang G, et al. Random erasing data augmentation[J]. arXiv preprint arXiv:1708.04896, 2017. + +[5] Singh K K, Lee Y J. Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization[C]//2017 IEEE international conference on computer vision (ICCV). IEEE, 2017: 3544-3553. + +[6] Chen P. GridMask Data Augmentation[J]. arXiv preprint arXiv:2001.04086, 2020. + +[7] Zhang H, Cisse M, Dauphin Y N, et al. mixup: Beyond empirical risk minimization[J]. arXiv preprint arXiv:1710.09412, 2017. + +[8] Yun S, Han D, Oh S J, et al. Cutmix: Regularization strategy to train strong classifiers with localizable features[C]//Proceedings of the IEEE International Conference on Computer Vision. 2019: 6023-6032. + + + +[test_baseline]: ../../images/image_aug/test_baseline.jpeg +[test_autoaugment]: ../../images/image_aug/test_autoaugment.jpeg +[test_cutout]: ../../images/image_aug/test_cutout.jpeg +[test_gridmask]: ../../images/image_aug/test_gridmask.jpeg +[gridmask-0]: ../../images/image_aug/gridmask-0.png +[test_hideandseek]: ../../images/image_aug/test_hideandseek.jpeg +[test_randaugment]: ../../images/image_aug/test_randaugment.jpeg +[test_randomerassing]: ../../images/image_aug/test_randomerassing.jpeg +[hide_and_seek_mask_expanation]: ../../images/image_aug/hide-and-seek-visual.png +[test_mixup]: ../../images/image_aug/test_mixup.png +[test_cutmix]: ../../images/image_aug/test_cutmix.png diff --git a/src/PaddleClas/docs/zh_CN/algorithm_introduction/ImageNet_models.md b/src/PaddleClas/docs/zh_CN/algorithm_introduction/ImageNet_models.md new file mode 100644 index 0000000..a61d882 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/algorithm_introduction/ImageNet_models.md @@ -0,0 +1,599 @@ + + + +# ImageNet 预训练模型库 + +## 目录 + +- [1. 模型库概览图](#1) +- [2. SSLD 知识蒸馏预训练模型](#2) + - [2.1 服务器端知识蒸馏模型](#2.1) + - [2.2 移动端知识蒸馏模型](#2.2) + - [2.3 Intel CPU 端知识蒸馏模型](#2.3) +- [3. PP-LCNet 系列](#3) +- [4. ResNet 系列](#4) +- [5. 移动端系列](#5) +- [6. SEResNeXt 与 Res2Net 系列](#6) +- [7. DPN 与 DenseNet 系列](#7) +- [8. HRNet 系列](#8) +- [9. Inception 系列](#9) +- [10. EfficientNet 与 ResNeXt101_wsl 系列](#10) +- [11. ResNeSt 与 RegNet 系列](#11) +- [12. ViT_and_DeiT 系列](#12) +- [13. RepVGG 系列](#13) +- [14. MixNet 系列](#14) +- [15. ReXNet 系列](#15) +- [16. SwinTransformer 系列](#16) +- [17. LeViT 系列](#17) +- [18. Twins 系列](#18) +- [19. HarDNet 系列](#19) +- [20. DLA 系列](#20) +- [21. RedNet 系列](#21) +- [22. TNT 系列](#22) +- [23. 其他模型](#23) +- [参考文献](#reference) + + + +## 1. 模型库概览图 + +基于 ImageNet1k 分类数据集，PaddleClas 支持 37 个系列分类网络结构以及对应的 217 个图像分类预训练模型，训练技巧、每个系列网络结构的简单介绍和性能评估将在相应章节展现，下面所有的速度指标评估环境如下： +* Arm CPU 的评估环境基于骁龙 855(SD855)。 +* Intel CPU 的评估环境基于 Intel(R) Xeon(R) Gold 6148。 +* GPU 评估环境基于 V100 机器，在 FP32+TensorRT 配置下运行 2100 次测得（去除前 100 次的 warmup 时间）。 +* FLOPs 与 Params 通过 `paddle.flops()` 计算得到（PaddlePaddle 版本为 2.2） + +常见服务器端模型的精度指标与其预测耗时的变化曲线如下图所示。 + +![](../../images/models/V100_benchmark/v100.fp32.bs1.main_fps_top1_s.png) + +常见移动端模型的精度指标与其预测耗时的变化曲线如下图所示。 + +![](../../images/models/mobile_arm_top1.png) + +部分VisionTransformer模型的精度指标与其预测耗时的变化曲线如下图所示. + +![](../../images/models/V100_benchmark/v100.fp32.bs1.visiontransformer.png) + + + +## 2. SSLD 知识蒸馏预训练模型 +基于 SSLD 知识蒸馏的预训练模型列表如下所示，更多关于 SSLD 知识蒸馏方案的介绍可以参考：[SSLD 知识蒸馏文档](./knowledge_distillation.md)。 + + + +### 2.1 服务器端知识蒸馏模型 + +| 模型 | Top-1 Acc | Reference
Top-1 Acc | Acc gain | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|---------------------|-----------|-----------|---------------|----------------|-----------|----------|-----------|-----------------------------------|-----------------------------------|-----------------------------------| +| ResNet34_vd_ssld | 0.797 | 0.760 | 0.037 | 2.00 | 3.28 | 5.84 | 3.93 | 21.84 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_vd_ssld_pretrained.pdparams)   | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_vd_ssld_infer.tar)   | +| ResNet50_vd_ssld | 0.830 | 0.792 | 0.039 | 2.60 | 4.86 | 7.63 | 4.35 | 25.63 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_ssld_infer.tar) | +| ResNet101_vd_ssld | 0.837 | 0.802 | 0.035 | 4.43 | 8.25 | 12.60 | 8.08 | 44.67 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_vd_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_vd_ssld_infer.tar) | +| Res2Net50_vd_26w_4s_ssld | 0.831 | 0.798 | 0.033 | 3.59 | 6.35 | 9.50 | 4.28 | 25.76 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_vd_26w_4s_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_vd_26w_4s_ssld_infer.tar) | +| Res2Net101_vd_
26w_4s_ssld | 0.839 | 0.806 | 0.033 | 6.34 | 11.02 | 16.13 | 8.35 | 45.35 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net101_vd_26w_4s_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net101_vd_26w_4s_ssld_infer.tar) | +| Res2Net200_vd_
26w_4s_ssld | 0.851 | 0.812 | 0.049 | 11.45 | 19.77 | 28.81 | 15.77 | 76.44 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net200_vd_26w_4s_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net200_vd_26w_4s_ssld_infer.tar) | +| HRNet_W18_C_ssld | 0.812 | 0.769 | 0.043 | 6.66 | 8.94 | 11.95 | 4.32 | 21.35 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W18_C_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W18_C_ssld_infer.tar) | +| HRNet_W48_C_ssld | 0.836 | 0.790 | 0.046 | 11.07 | 17.06 | 27.28 | 17.34 | 77.57 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W48_C_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W48_C_ssld_infer.tar) | +| SE_HRNet_W64_C_ssld | 0.848 | - | - | 17.11 | 26.87 | 43.24 | 29.00 | 129.12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/SE_HRNet_W64_C_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_HRNet_W64_C_ssld_infer.tar) | + + + +### 2.2 移动端知识蒸馏模型 + +| 模型 | Top-1 Acc | Reference
Top-1 Acc | Acc gain | SD855 time(ms)
bs=1, thread=1 | SD855 time(ms)
bs=1, thread=2 | SD855 time(ms)
bs=1, thread=4 | FLOPs(M) | Params(M) | 模型大小(M) | 预训练模型下载地址 | inference模型下载地址 | +|---------------------|-----------|-----------|---------------|----------------|-----------|----------|-----------|-----------------------------------|-----------------------------------|-----------------------------------|-----------------------------------| +| MobileNetV1_ssld | 0.779 | 0.710 | 0.069 | 30.24 | 17.86 | 10.30 | 578.88 | 4.25 | 16 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_ssld_infer.tar) | +| MobileNetV2_ssld | 0.767 | 0.722 | 0.045 | 20.74 | 12.71 | 8.10 | 327.84 | 3.54 | 14 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_ssld_infer.tar) | +| MobileNetV3_small_x0_35_ssld | 0.556 | 0.530 | 0.026 | 2.23 | 1.66 | 1.43 | 14.56 | 1.67 | 6.9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_35_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_35_ssld_infer.tar) | +| MobileNetV3_large_x1_0_ssld | 0.790 | 0.753 | 0.036 | 16.55 | 10.09 | 6.84 | 229.66 | 5.50 | 21 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_0_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_ssld_infer.tar) | +| MobileNetV3_small_x1_0_ssld | 0.713 | 0.682 | 0.031 | 5.63 | 3.65 | 2.60 | 63.67 | 2.95 | 12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_0_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_0_ssld_infer.tar) | +| GhostNet_x1_3_ssld | 0.794 | 0.757 | 0.037 | 19.16 | 12.25 | 9.40 | 236.89 | 7.38 | 29 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_3_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_3_ssld_infer.tar) | + + + +### 2.3 Intel CPU 端知识蒸馏模型 + +| 模型 | Top-1 Acc | Reference
Top-1 Acc | Acc gain | Intel-Xeon-Gold-6148 time(ms)
bs=1 | FLOPs(M) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|---------------------|-----------|-----------|---------------|----------------|----------|-----------|-----------------------------------|-----------------------------------| +| PPLCNet_x0_5_ssld | 0.661 | 0.631 | 0.030 | 2.05 | 47.28 | 1.89 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_5_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_5_ssld_infer.tar) | +| PPLCNet_x1_0_ssld | 0.744 | 0.713 | 0.033 | 2.46 | 160.81 | 2.96 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x1_0_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x1_0_ssld_infer.tar) | +| PPLCNet_x2_5_ssld | 0.808 | 0.766 | 0.042 | 5.39 | 906.49 | 9.04 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x2_5_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x2_5_ssld_infer.tar) | + +* 注: `Reference Top-1 Acc` 表示 PaddleClas 基于 ImageNet1k 数据集训练得到的预训练模型精度。 + + + +## 3. PP-LCNet 系列 [[28](#ref28)] + +PP-LCNet 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[PP-LCNet 系列模型文档](../models/PP-LCNet.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | Intel-Xeon-Gold-6148 time(ms)
bs=1 | FLOPs(M) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|:--:|:--:|:--:|:--:|----|----|----|:--:| +| PPLCNet_x0_25 |0.5186 | 0.7565 | 1.61785 | 18.25 | 1.52 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_25_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_25_infer.tar) | +| PPLCNet_x0_35 |0.5809 | 0.8083 | 2.11344 | 29.46 | 1.65 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_35_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_35_infer.tar) | +| PPLCNet_x0_5 |0.6314 | 0.8466 | 2.72974 | 47.28 | 1.89 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_5_infer.tar) | +| PPLCNet_x0_75 |0.6818 | 0.8830 | 4.51216 | 98.82 | 2.37 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x0_75_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x0_75_infer.tar) | +| PPLCNet_x1_0 |0.7132 | 0.9003 | 6.49276 | 160.81 | 2.96 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x1_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x1_0_infer.tar) | +| PPLCNet_x1_5 |0.7371 | 0.9153 | 12.2601 | 341.86 | 4.52 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x1_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x1_5_infer.tar) | +| PPLCNet_x2_0 |0.7518 | 0.9227 | 20.1667 | 590 | 6.54 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x2_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x2_0_infer.tar) | +| PPLCNet_x2_5 |0.7660 | 0.9300 | 29.595 | 906 | 9.04 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/PPLCNet_x2_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/PPLCNet_x2_5_infer.tar) | + + + +## 4. ResNet 系列 [[1](#ref1)] + +ResNet 及其 Vd 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[ResNet 及其 Vd 系列模型文档](../models/ResNet_and_vd.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|---------------------|-----------|-----------|-----------------------|----------------------|----------|-----------|----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------| +| ResNet18 | 0.7098 | 0.8992 | 1.22 | 2.19 | 3.63 | 1.83 | 11.70 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet18_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet18_infer.tar) | +| ResNet18_vd | 0.7226 | 0.9080 | 1.26 | 2.28 | 3.89 | 2.07 | 11.72 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet18_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet18_vd_infer.tar) | +| ResNet34 | 0.7457 | 0.9214 | 1.97 | 3.25 | 5.70 | 3.68 | 21.81 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_infer.tar) | +| ResNet34_vd | 0.7598 | 0.9298 | 2.00 | 3.28 | 5.84 | 3.93 | 21.84 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_vd_infer.tar) | +| ResNet34_vd_ssld | 0.7972 | 0.9490 | 2.00 | 3.28 | 5.84 | 3.93 | 21.84 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet34_vd_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet34_vd_ssld_infer.tar) | +| ResNet50 | 0.7650 | 0.9300 | 2.54 | 4.79 | 7.40 | 4.11 | 25.61 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_infer.tar) | +| ResNet50_vc | 0.7835 | 0.9403 | 2.57 | 4.83 | 7.52 | 4.35 | 25.63 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNet50_vc_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vc_infer.tar) | +| ResNet50_vd | 0.7912 | 0.9444 | 2.60 | 4.86 | 7.63 | 4.35 | 25.63 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar) | +| ResNet101 | 0.7756 | 0.9364 | 4.37 | 8.18 | 12.38 | 7.83 | 44.65 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_infer.tar) | +| ResNet101_vd | 0.8017 | 0.9497 | 4.43 | 8.25 | 12.60 | 8.08 | 44.67 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_vd_infer.tar) | +| ResNet152 | 0.7826 | 0.9396 | 6.05 | 11.41 | 17.33 | 11.56 | 60.34 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet152_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet152_infer.tar) | +| ResNet152_vd | 0.8059 | 0.9530 | 6.11 | 11.51 | 17.59 | 11.80 | 60.36 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet152_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet152_vd_infer.tar) | +| ResNet200_vd | 0.8093 | 0.9533 | 7.70 | 14.57 | 22.16 | 15.30 | 74.93 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet200_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet200_vd_infer.tar) | +| ResNet50_vd_
ssld | 0.8300 | 0.9640 | 2.60 | 4.86 | 7.63 | 4.35 | 25.63 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_ssld_infer.tar) | +| ResNet101_vd_
ssld | 0.8373 | 0.9669 | 4.43 | 8.25 | 12.60 | 8.08 | 44.67 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet101_vd_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet101_vd_ssld_infer.tar) | + + + +## 5. 移动端系列 [[3](#ref3)][[4](#ref4)][[5](#ref5)][[6](#ref6)][[23](#ref23)] + +移动端系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[移动端系列模型文档](../models/Mobile.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | SD855 time(ms)
bs=1, thread=1 | SD855 time(ms)
bs=1, thread=2 | SD855 time(ms)
bs=1, thread=4 | FLOPs(M) | Params(M) | 模型大小(M) | 预训练模型下载地址 | inference模型下载地址 | +|----------------------------------|-----------|-----------|------------------------|----------|-----------|---------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------| +| MobileNetV1_
x0_25 | 0.5143 | 0.7546 | 2.88 | 1.82 | 1.26 | 43.56 | 0.48 | 1.9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_x0_25_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_x0_25_infer.tar) | +| MobileNetV1_
x0_5 | 0.6352 | 0.8473 | 8.74 | 5.26 | 3.09 | 154.57 | 1.34 | 5.2 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_x0_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_x0_5_infer.tar) | +| MobileNetV1_
x0_75 | 0.6881 | 0.8823 | 17.84 | 10.61 | 6.21 | 333.00 | 2.60 | 10 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_x0_75_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_x0_75_infer.tar) | +| MobileNetV1 | 0.7099 | 0.8968 | 30.24 | 17.86 | 10.30 | 578.88 | 4.25 | 16 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_infer.tar) | +| MobileNetV1_
ssld | 0.7789 | 0.9394 | 30.24 | 17.86 | 10.30 | 578.88 | 4.25 | 16 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV1_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV1_ssld_infer.tar) | +| MobileNetV2_
x0_25 | 0.5321 | 0.7652 | 3.46 | 2.51 | 2.03 | 34.18 | 1.53 | 6.1 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x0_25_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x0_25_infer.tar) | +| MobileNetV2_
x0_5 | 0.6503 | 0.8572 | 7.69 | 4.92 | 3.57 | 99.48 | 1.98 | 7.8 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x0_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x0_5_infer.tar) | +| MobileNetV2_
x0_75 | 0.6983 | 0.8901 | 13.69 | 8.60 | 5.82 | 197.37 | 2.65 | 10 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x0_75_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x0_75_infer.tar) | +| MobileNetV2 | 0.7215 | 0.9065 | 20.74 | 12.71 | 8.10 | 327.84 | 3.54 | 14 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_infer.tar) | +| MobileNetV2_
x1_5 | 0.7412 | 0.9167 | 40.79 | 24.49 | 15.50 | 702.35 | 6.90 | 26 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x1_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x1_5_infer.tar) | +| MobileNetV2_
x2_0 | 0.7523 | 0.9258 | 67.50 | 40.03 | 25.55 | 1217.25 | 11.33 | 43 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_x2_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_x2_0_infer.tar) | +| MobileNetV2_
ssld | 0.7674 | 0.9339 | 20.74 | 12.71 | 8.10 | 327.84 | 3.54 | 14 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV2_ssld_infer.tar) | +| MobileNetV3_
large_x1_25 | 0.7641 | 0.9295 | 24.52 | 14.76 | 9.89 | 362.70 | 7.47 | 29 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_25_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_25_infer.tar) | +| MobileNetV3_
large_x1_0 | 0.7532 | 0.9231 | 16.55 | 10.09 | 6.84 | 229.66 | 5.50 | 21 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_infer.tar) | +| MobileNetV3_
large_x0_75 | 0.7314 | 0.9108 | 11.53 | 7.06 | 4.94 | 151.70 | 3.93 | 16 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x0_75_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x0_75_infer.tar) | +| MobileNetV3_
large_x0_5 | 0.6924 | 0.8852 | 6.50 | 4.22 | 3.15 | 71.83 | 2.69 | 11 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x0_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x0_5_infer.tar) | +| MobileNetV3_
large_x0_35 | 0.6432 | 0.8546 | 4.43 | 3.11 | 2.41 | 40.90 | 2.11 | 8.6 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x0_35_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x0_35_infer.tar) | +| MobileNetV3_
small_x1_25 | 0.7067 | 0.8951 | 7.88 | 4.91 | 3.45 | 100.07 | 3.64 | 14 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_25_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_25_infer.tar) | +| MobileNetV3_
small_x1_0 | 0.6824 | 0.8806 | 5.63 | 3.65 | 2.60 | 63.67 | 2.95 | 12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_0_infer.tar) | +| MobileNetV3_
small_x0_75 | 0.6602 | 0.8633 | 4.50 | 2.96 | 2.19 | 46.02 | 2.38 | 9.6 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_75_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_75_infer.tar) | +| MobileNetV3_
small_x0_5 | 0.5921 | 0.8152 | 2.89 | 2.04 | 1.62 | 22.60 | 1.91 | 7.8 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_5_infer.tar) | +| MobileNetV3_
small_x0_35 | 0.5303 | 0.7637 | 2.23 | 1.66 | 1.43 | 14.56 | 1.67 | 6.9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_35_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_35_infer.tar) | +| MobileNetV3_
small_x0_35_ssld | 0.5555 | 0.7771 | 2.23 | 1.66 | 1.43 | 14.56 | 1.67 | 6.9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x0_35_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x0_35_ssld_infer.tar) | +| MobileNetV3_
large_x1_0_ssld | 0.7896 | 0.9448 | 16.55 | 10.09 | 6.84 | 229.66 | 5.50 | 21 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_large_x1_0_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_ssld_infer.tar) | +| MobileNetV3_small_
x1_0_ssld | 0.7129 | 0.9010 | 5.63 | 3.65 | 2.60 | 63.67 | 2.95 | 12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/MobileNetV3_small_x1_0_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_small_x1_0_ssld_infer.tar) | +| ShuffleNetV2 | 0.6880 | 0.8845 | 9.72 | 5.97 | 4.13 | 148.86 | 2.29 | 9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x1_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x1_0_infer.tar) | +| ShuffleNetV2_
x0_25 | 0.4990 | 0.7379 | 1.94 | 1.53 | 1.43 | 18.95 | 0.61 | 2.7 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x0_25_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x0_25_infer.tar) | +| ShuffleNetV2_
x0_33 | 0.5373 | 0.7705 | 2.23 | 1.70 | 1.79 | 24.04 | 0.65 | 2.8 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x0_33_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x0_33_infer.tar) | +| ShuffleNetV2_
x0_5 | 0.6032 | 0.8226 | 3.67 | 2.63 | 2.06 | 42.58 | 1.37 | 5.6 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x0_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x0_5_infer.tar) | +| ShuffleNetV2_
x1_5 | 0.7163 | 0.9015 | 17.21 | 10.56 | 6.81 | 301.35 | 3.53 | 14 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x1_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x1_5_infer.tar) | +| ShuffleNetV2_
x2_0 | 0.7315 | 0.9120 | 31.21 | 18.98 | 11.65 | 571.70 | 7.40 | 28 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_x2_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_x2_0_infer.tar) | +| ShuffleNetV2_
swish | 0.7003 | 0.8917 | 31.21 | 9.06 | 5.74 | 148.86 | 2.29 | 9.1 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ShuffleNetV2_swish_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ShuffleNetV2_swish_infer.tar) | +| GhostNet_
x0_5 | 0.6688 | 0.8695 | 5.28 | 3.95 | 3.29 | 46.15 | 2.60 | 10 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x0_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x0_5_infer.tar) | +| GhostNet_
x1_0 | 0.7402 | 0.9165 | 12.89 | 8.66 | 6.72 | 148.78 | 5.21 | 20 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_0_infer.tar) | +| GhostNet_
x1_3 | 0.7579 | 0.9254 | 19.16 | 12.25 | 9.40 | 236.89 | 7.38 | 29 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_3_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_3_infer.tar) | +| GhostNet_
x1_3_ssld | 0.7938 | 0.9449 | 19.16 | 12.25 | 9.40 | 236.89 | 7.38 | 29 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GhostNet_x1_3_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GhostNet_x1_3_ssld_infer.tar) | +| ESNet_x0_25 | 0.6248 | 0.8346 |4.12|2.97|2.51| 30.85 | 2.83 | 11 |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x0_25_pretrained.pdparams) |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x0_25_infer.tar) | +| ESNet_x0_5 | 0.6882 | 0.8804 |6.45|4.42|3.35| 67.31 | 3.25 | 13 |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x0_5_pretrained.pdparams) |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x0_5_infer.tar) | +| ESNet_x0_75 | 0.7224 | 0.9045 |9.59|6.28|4.52| 123.74 | 3.87 | 15 |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x0_75_pretrained.pdparams) |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x0_75_infer.tar) | +| ESNet_x1_0 | 0.7392 | 0.9140 |13.67|8.71|5.97| 197.33 | 4.64 | 18 |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ESNet_x1_0_pretrained.pdparams) |[下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ESNet_x1_0_infer.tar) | + + + +## 6. SEResNeXt 与 Res2Net 系列 [[7](#ref7)][[8](#ref8)][[9](#ref9)] + +SEResNeXt 与 Res2Net 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[SEResNeXt 与 Res2Net 系列模型文档](../models/SEResNext_and_Res2Net.md)。 + + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|---------------------------|-----------|-----------|-----------------------|----------------------|----------|-----------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------| +| Res2Net50_
26w_4s | 0.7933 | 0.9457 | 3.52 | 6.23 | 9.30 | 4.28 | 25.76 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_26w_4s_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_26w_4s_infer.tar) | +| Res2Net50_vd_
26w_4s | 0.7975 | 0.9491 | 3.59 | 6.35 | 9.50 | 4.52 | 25.78 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_vd_26w_4s_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_vd_26w_4s_infer.tar) | +| Res2Net50_
14w_8s | 0.7946 | 0.9470 | 4.39 | 7.21 | 10.38 | 4.20 | 25.12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net50_14w_8s_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net50_14w_8s_infer.tar) | +| Res2Net101_vd_
26w_4s | 0.8064 | 0.9522 | 6.34 | 11.02 | 16.13 | 8.35 | 45.35 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net101_vd_26w_4s_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net101_vd_26w_4s_infer.tar) | +| Res2Net200_vd_
26w_4s | 0.8121 | 0.9571 | 11.45 | 19.77 | 28.81 | 15.77 | 76.44 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net200_vd_26w_4s_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net200_vd_26w_4s_infer.tar) | +| Res2Net200_vd_
26w_4s_ssld | 0.8513 | 0.9742 | 11.45 | 19.77 | 28.81 | 15.77 | 76.44 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Res2Net200_vd_26w_4s_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Res2Net200_vd_26w_4s_ssld_infer.tar) | +| ResNeXt50_
32x4d | 0.7775 | 0.9382 | 5.07 | 8.49 | 12.02 | 4.26 | 25.10 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_32x4d_infer.tar) | +| ResNeXt50_vd_
32x4d | 0.7956 | 0.9462 | 5.29 | 8.68 | 12.33 | 4.50 | 25.12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_vd_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_vd_32x4d_infer.tar) | +| ResNeXt50_
64x4d | 0.7843 | 0.9413 | 9.39 | 13.97 | 20.56 | 8.02 | 45.29 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_64x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_64x4d_infer.tar) | +| ResNeXt50_vd_
64x4d | 0.8012 | 0.9486 | 9.75 | 14.14 | 20.84 | 8.26 | 45.31 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt50_vd_64x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt50_vd_64x4d_infer.tar) | +| ResNeXt101_
32x4d | 0.7865 | 0.9419 | 11.34 | 16.78 | 22.80 | 8.01 | 44.32 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x4d_infer.tar) | +| ResNeXt101_vd_
32x4d | 0.8033 | 0.9512 | 11.36 | 17.01 | 23.07 | 8.25 | 44.33 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_vd_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_vd_32x4d_infer.tar) | +| ResNeXt101_
64x4d | 0.7835 | 0.9452 | 21.57 | 28.08 | 39.49 | 15.52 | 83.66 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_64x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_64x4d_infer.tar) | +| ResNeXt101_vd_
64x4d | 0.8078 | 0.9520 | 21.57 | 28.22 | 39.70 | 15.76 | 83.68 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_vd_64x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_vd_64x4d_infer.tar) | +| ResNeXt152_
32x4d | 0.7898 | 0.9433 | 17.14 | 25.11 | 33.79 | 11.76 | 60.15 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_32x4d_infer.tar) | +| ResNeXt152_vd_
32x4d | 0.8072 | 0.9520 | 16.99 | 25.29 | 33.85 | 12.01 | 60.17 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_vd_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_vd_32x4d_infer.tar) | +| ResNeXt152_
64x4d | 0.7951 | 0.9471 | 33.07 | 42.05 | 59.13 | 23.03 | 115.27 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_64x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_64x4d_infer.tar) | +| ResNeXt152_vd_
64x4d | 0.8108 | 0.9534 | 33.30 | 42.41 | 59.42 | 23.27 | 115.29 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt152_vd_64x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt152_vd_64x4d_infer.tar) | +| SE_ResNet18_vd | 0.7333 | 0.9138 | 1.48 | 2.70 | 4.32 | 2.07 | 11.81 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNet18_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNet18_vd_infer.tar) | +| SE_ResNet34_vd | 0.7651 | 0.9320 | 2.42 | 3.69 | 6.29 | 3.93 | 22.00 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNet34_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNet34_vd_infer.tar) | +| SE_ResNet50_vd | 0.7952 | 0.9475 | 3.11 | 5.99 | 9.34 | 4.36 | 28.16 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNet50_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNet50_vd_infer.tar) | +| SE_ResNeXt50_
32x4d | 0.7844 | 0.9396 | 6.39 | 11.01 | 14.94 | 4.27 | 27.63 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNeXt50_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNeXt50_32x4d_infer.tar) | +| SE_ResNeXt50_vd_
32x4d | 0.8024 | 0.9489 | 7.04 | 11.57 | 16.01 | 5.64 | 27.76 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNeXt50_vd_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNeXt50_vd_32x4d_infer.tar) | +| SE_ResNeXt101_
32x4d | 0.7939 | 0.9443 | 13.31 | 21.85 | 28.77 | 8.03 | 49.09 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SE_ResNeXt101_32x4d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_ResNeXt101_32x4d_infer.tar) | +| SENet154_vd | 0.8140 | 0.9548 | 34.83 | 51.22 | 69.74 | 24.45 | 122.03 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SENet154_vd_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SENet154_vd_infer.tar) | + + + +## 7. DPN 与 DenseNet 系列 [[14](#ref14)][[15](#ref15)] + +DPN 与 DenseNet 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[DPN 与 DenseNet 系列模型文档](../models/DPN_DenseNet.md)。 + + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|-------------|-----------|-----------|-----------------------|----------------------|----------|-----------|--------------------------------------------------------------------------------------|-------------|-------------| +| DenseNet121 | 0.7566 | 0.9258 | 3.40 | 6.94 | 9.17 | 2.87 | 8.06 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet121_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet121_infer.tar) | +| DenseNet161 | 0.7857 | 0.9414 | 7.06 | 14.37 | 19.55 | 7.79 | 28.90 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet161_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet161_infer.tar) | +| DenseNet169 | 0.7681 | 0.9331 | 5.00 | 10.29 | 12.84 | 3.40 | 14.31 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet169_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet169_infer.tar) | +| DenseNet201 | 0.7763 | 0.9366 | 6.38 | 13.72 | 17.17 | 4.34 | 20.24 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet201_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet201_infer.tar) | +| DenseNet264 | 0.7796 | 0.9385 | 9.34 | 20.95 | 25.41 | 5.82 | 33.74 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DenseNet264_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DenseNet264_infer.tar) | +| DPN68 | 0.7678 | 0.9343 | 8.18 | 11.40 | 14.82 | 2.35 | 12.68 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN68_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN68_infer.tar) | +| DPN92 | 0.7985 | 0.9480 | 12.48 | 20.04 | 25.10 | 6.54 | 37.79 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN92_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN92_infer.tar) | +| DPN98 | 0.8059 | 0.9510 | 14.70 | 25.55 | 35.12 | 11.728 | 61.74 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN98_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN98_infer.tar) | +| DPN107 | 0.8089 | 0.9532 | 19.46 | 35.62 | 50.22 | 18.38 | 87.13 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN107_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN107_infer.tar) | +| DPN131 | 0.8070 | 0.9514 | 19.64 | 34.60 | 47.42 | 16.09 | 79.48 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DPN131_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DPN131_infer.tar) | + + + +## 8. HRNet 系列 [[13](#ref13)] + +HRNet 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[HRNet 系列模型文档](../models/HRNet.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|-------------|-----------|-----------|------------------|------------------|----------|-----------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------| +| HRNet_W18_C | 0.7692 | 0.9339 | 6.66 | 8.94 | 11.95 | 4.32 | 21.35 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W18_C_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W18_C_infer.tar) | +| HRNet_W18_C_ssld | 0.81162 | 0.95804 | 6.66 | 8.94 | 11.95 | 4.32 | 21.35 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W18_C_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W18_C_ssld_infer.tar) | +| HRNet_W30_C | 0.7804 | 0.9402 | 8.61 | 11.40 | 15.23 | 8.15 | 37.78 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W30_C_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W30_C_infer.tar) | +| HRNet_W32_C | 0.7828 | 0.9424 | 8.54 | 11.58 | 15.57 | 8.97 | 41.30 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W32_C_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W32_C_infer.tar) | +| HRNet_W40_C | 0.7877 | 0.9447 | 9.83 | 15.02 | 20.92 | 12.74 | 57.64 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W40_C_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W40_C_infer.tar) | +| HRNet_W44_C | 0.7900 | 0.9451 | 10.62 | 16.18 | 25.92 | 14.94 | 67.16 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W44_C_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W44_C_infer.tar) | +| HRNet_W48_C | 0.7895 | 0.9442 | 11.07 | 17.06 | 27.28 | 17.34 | 77.57 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W48_C_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W48_C_infer.tar) | +| HRNet_W48_C_ssld | 0.8363 | 0.9682 | 11.07 | 17.06 | 27.28 | 17.34 | 77.57 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W48_C_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W48_C_ssld_infer.tar) | +| HRNet_W64_C | 0.7930 | 0.9461 | 13.82 | 21.15 | 35.51 | 28.97 | 128.18 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/HRNet_W64_C_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HRNet_W64_C_infer.tar) | +| SE_HRNet_W64_C_ssld | 0.8475 | 0.9726 | 17.11 | 26.87 | 43.24 | 29.00 | 129.12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/SE_HRNet_W64_C_ssld_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SE_HRNet_W64_C_ssld_infer.tar) | + + + +## 9. Inception 系列 [[10](#ref10)][[11](#ref11)][[12](#ref12)][[26](#ref26)] + +Inception 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[Inception 系列模型文档](../models/Inception.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|--------------------|-----------|-----------|-----------------------|----------------------|----------|-----------|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------| +| GoogLeNet | 0.7070 | 0.8966 | 1.41 | 3.25 | 5.00 | 1.44 | 11.54 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/GoogLeNet_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/GoogLeNet_infer.tar) | +| Xception41 | 0.7930 | 0.9453 | 3.58 | 8.76 | 16.61 | 8.57 | 23.02 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception41_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception41_infer.tar) | +| Xception41_deeplab | 0.7955 | 0.9438 | 3.81 | 9.16 | 17.20 | 9.28 | 27.08 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception41_deeplab_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception41_deeplab_infer.tar) | +| Xception65 | 0.8100 | 0.9549 | 5.45 | 12.78 | 24.53 | 13.25 | 36.04 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception65_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception65_infer.tar) | +| Xception65_deeplab | 0.8032 | 0.9449 | 5.65 | 13.08 | 24.61 | 13.96 | 40.10 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception65_deeplab_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception65_deeplab_infer.tar) | +| Xception71 | 0.8111 | 0.9545 | 6.19 | 15.34 | 29.21 | 16.21 | 37.86 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Xception71_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Xception71_infer.tar) | +| InceptionV3 | 0.7914 | 0.9459 | 4.78 | 8.53 | 12.28 | 5.73 | 23.87 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/InceptionV3_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/InceptionV3_infer.tar) | +| InceptionV4 | 0.8077 | 0.9526 | 8.93 | 15.17 | 21.56 | 12.29 | 42.74 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/InceptionV4_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/InceptionV4_infer.tar) | + + + +## 10. EfficientNet 与 ResNeXt101_wsl 系列 [[16](#ref16)][[17](#ref17)] + +EfficientNet 与 ResNeXt101_wsl 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[EfficientNet 与 ResNeXt101_wsl 系列模型文档](../models/EfficientNet_and_ResNeXt101_wsl.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|---------------------------|-----------|-----------|------------------|------------------|----------|-----------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------| +| ResNeXt101_
32x8d_wsl | 0.8255 | 0.9674 | 13.55 | 23.39 | 36.18 | 16.48 | 88.99 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x8d_wsl_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x8d_wsl_infer.tar) | +| ResNeXt101_
32x16d_wsl | 0.8424 | 0.9726 | 21.96 | 38.35 | 63.29 | 36.26 | 194.36 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x16d_wsl_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x16d_wsl_infer.tar) | +| ResNeXt101_
32x32d_wsl | 0.8497 | 0.9759 | 37.28 | 76.50 | 121.56 | 87.28 | 469.12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x32d_wsl_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x32d_wsl_infer.tar) | +| ResNeXt101_
32x48d_wsl | 0.8537 | 0.9769 | 55.07 | 124.39 | 205.01 | 153.57 | 829.26 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeXt101_32x48d_wsl_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeXt101_32x48d_wsl_infer.tar) | +| Fix_ResNeXt101_
32x48d_wsl | 0.8626 | 0.9797 | 55.01 | 122.63 | 204.66 | 313.41 | 829.26 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/Fix_ResNeXt101_32x48d_wsl_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/Fix_ResNeXt101_32x48d_wsl_infer.tar) | +| EfficientNetB0 | 0.7738 | 0.9331 | 1.96 | 3.71 | 5.56 | 0.40 | 5.33 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB0_infer.tar) | +| EfficientNetB1 | 0.7915 | 0.9441 | 2.88 | 5.40 | 7.63 | 0.71 | 7.86 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB1_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB1_infer.tar) | +| EfficientNetB2 | 0.7985 | 0.9474 | 3.26 | 6.20 | 9.17 | 1.02 | 9.18 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB2_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB2_infer.tar) | +| EfficientNetB3 | 0.8115 | 0.9541 | 4.52 | 8.85 | 13.54 | 1.88 | 12.324 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB3_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB3_infer.tar) | +| EfficientNetB4 | 0.8285 | 0.9623 | 6.78 | 15.47 | 24.95 | 4.51 | 19.47 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB4_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB4_infer.tar) | +| EfficientNetB5 | 0.8362 | 0.9672 | 10.97 | 27.24 | 45.93 | 10.51 | 30.56 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB5_infer.tar) | +| EfficientNetB6 | 0.8400 | 0.9688 | 17.09 | 43.32 | 76.90 | 19.47 | 43.27 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB6_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB6_infer.tar) | +| EfficientNetB7 | 0.8430 | 0.9689 | 25.91 | 71.23 | 128.20 | 38.45 | 66.66 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB7_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB7_infer.tar) | +| EfficientNetB0_
small | 0.7580 | 0.9258 | 1.24 | 2.59 | 3.92 | 0.40 | 4.69 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/EfficientNetB0_small_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB0_small_infer.tar) | + + + +## 11. ResNeSt 与 RegNet 系列 [[24](#ref24)][[25](#ref25)] + +ResNeSt 与 RegNet 系列模型的精度、速度指标如下表所示，更多关于该系列的模型介绍可以参考：[ResNeSt 与 RegNet 系列模型文档](../models/ResNeSt_RegNet.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|------------------------|-----------|-----------|------------------|------------------|----------|-----------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| +| ResNeSt50_
fast_1s1x64d | 0.8035 | 0.9528 | 2.73 | 5.33 | 8.24 | 4.36 | 26.27 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeSt50_fast_1s1x64d_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeSt50_fast_1s1x64d_infer.tar) | +| ResNeSt50 | 0.8083 | 0.9542 | 7.36 | 10.23 | 13.84 | 5.40 | 27.54 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNeSt50_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNeSt50_infer.tar) | +| RegNetX_4GF | 0.785 | 0.9416 | 6.46 | 8.48 | 11.45 | 4.00 | 22.23 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RegNetX_4GF_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RegNetX_4GF_infer.tar) | + + + +## 12. ViT_and_DeiT 系列 [[31](#ref31)][[32](#ref32)] + +ViT(Vision Transformer) 与 DeiT（Data-efficient Image Transformers）系列模型的精度、速度指标如下表所示. 更多关于该系列模型的介绍可以参考： [ViT_and_DeiT 系列模型文档](../models/ViT_and_DeiT.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|------------------------|-----------|-----------|------------------|------------------|----------|------------------------|------------------------|------------------------|------------------------| +| ViT_small_
patch16_224 | 0.7769 | 0.9342 | 3.71 | 9.05 | 16.72 | 9.41 | 48.60 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_small_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_small_patch16_224_infer.tar) | +| ViT_base_
patch16_224 | 0.8195 | 0.9617 | 6.12 | 14.84 | 28.51 | 16.85 | 86.42 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_base_patch16_224_infer.tar) | +| ViT_base_
patch16_384 | 0.8414 | 0.9717 | 14.15 | 48.38 | 95.06 | 49.35 | 86.42 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch16_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_base_patch16_384_infer.tar) | +| ViT_base_
patch32_384 | 0.8176 | 0.9613 | 4.94 | 13.43 | 24.08 | 12.66 | 88.19 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch32_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_base_patch32_384_infer.tar) | +| ViT_large_
patch16_224 | 0.8323 | 0.9650 | 15.53 | 49.50 | 94.09 | 59.65 | 304.12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_large_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_large_patch16_224_infer.tar) | +|ViT_large_
patch16_384| 0.8513 | 0.9736 | 39.51 | 152.46 | 304.06 | 174.70 | 304.12 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_large_patch16_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_large_patch16_384_infer.tar) | +|ViT_large_
patch32_384| 0.8153 | 0.9608 | 11.44 | 36.09 | 70.63 | 44.24 | 306.48 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_large_patch32_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ViT_large_patch32_384_infer.tar) | + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|------------------------|-----------|-----------|------------------|------------------|----------|------------------------|------------------------|------------------------|------------------------| +| DeiT_tiny_
patch16_224 | 0.718 | 0.910 | 3.61 | 3.94 | 6.10 | 1.07 | 5.68 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_tiny_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_tiny_patch16_224_infer.tar) | +| DeiT_small_
patch16_224 | 0.796 | 0.949 | 3.61 | 6.24 | 10.49 | 4.24 | 21.97 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_small_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_small_patch16_224_infer.tar) | +| DeiT_base_
patch16_224 | 0.817 | 0.957 | 6.13 | 14.87 | 28.50 | 16.85 | 86.42 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_patch16_224_infer.tar) | +| DeiT_base_
patch16_384 | 0.830 | 0.962 | 14.12 | 48.80 | 97.60 | 49.35 | 86.42 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_patch16_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_patch16_384_infer.tar) | +| DeiT_tiny_
distilled_patch16_224 | 0.741 | 0.918 | 3.51 | 4.05 | 6.03 | 1.08 | 5.87 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_tiny_distilled_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_tiny_distilled_patch16_224_infer.tar) | +| DeiT_small_
distilled_patch16_224 | 0.809 | 0.953 | 3.70 | 6.20 | 10.53 | 4.26 | 22.36 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_small_distilled_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_small_distilled_patch16_224_infer.tar) | +| DeiT_base_
distilled_patch16_224 | 0.831 | 0.964 | 6.17 | 14.94 | 28.58 | 16.93 | 87.18 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_distilled_patch16_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_distilled_patch16_224_infer.tar) | +| DeiT_base_
distilled_patch16_384 | 0.851 | 0.973 | 14.12 | 48.76 | 97.09 | 49.43 | 87.18 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DeiT_base_distilled_patch16_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DeiT_base_distilled_patch16_384_infer.tar) | + + + +## 13. RepVGG 系列 [[36](#ref36)] + +关于 RepVGG 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[RepVGG 系列模型文档](../models/RepVGG.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|------------------------|-----------|-----------|------------------|------------------|----------|-----------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| +| RepVGG_A0 | 0.7131 | 0.9016 | | | | 1.36 | 8.31 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_A0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_A0_infer.tar) | +| RepVGG_A1 | 0.7380 | 0.9146 | | | | 2.37 | 12.79 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_A1_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_A1_infer.tar) | +| RepVGG_A2 | 0.7571 | 0.9264 | | | | 5.12 | 25.50 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_A2_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_A2_infer.tar) | +| RepVGG_B0 | 0.7450 | 0.9213 | | | | 3.06 | 14.34 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B0_infer.tar) | +| RepVGG_B1 | 0.7773 | 0.9385 | | | | 11.82 | 51.83 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B1_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B1_infer.tar) | +| RepVGG_B2 | 0.7813 | 0.9410 | | | | 18.38 | 80.32 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B2_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B2_infer.tar) | +| RepVGG_B1g2 | 0.7732 | 0.9359 | | | | 8.82 | 41.36 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B1g2_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B1g2_infer.tar) | +| RepVGG_B1g4 | 0.7675 | 0.9335 | | | | 7.31 | 36.13 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B1g4_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B1g4_infer.tar) | +| RepVGG_B2g4 | 0.7881 | 0.9448 | | | | 11.34 | 55.78 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B2g4_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B2g4_infer.tar) | +| RepVGG_B3g4 | 0.7965 | 0.9485 | | | | 16.07 | 75.63 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RepVGG_B3g4_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RepVGG_B3g4_infer.tar) | + + + +## 14. MixNet 系列 [[29](#ref29)] + +关于 MixNet 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[MixNet 系列模型文档](../models/MixNet.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(M) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| -------- | --------- | --------- | ---------------- | ---------------- | ----------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| MixNet_S | 0.7628 | 0.9299 | 2.31 | 3.63 | 5.20 | 252.977 | 4.167 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MixNet_S_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MixNet_S_infer.tar) | +| MixNet_M | 0.7767 | 0.9364 | 2.84 | 4.60 | 6.62 | 357.119 | 5.065 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MixNet_M_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MixNet_M_infer.tar) | +| MixNet_L | 0.7860 | 0.9437 | 3.16 | 5.55 | 8.03 | 579.017 | 7.384 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MixNet_L_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MixNet_L_infer.tar) | + + + +## 15. ReXNet 系列 [[30](#ref30)] + +关于 ReXNet 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[ReXNet 系列模型文档](../models/ReXNet.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| ReXNet_1_0 | 0.7746 | 0.9370 | 3.08 | 4.15 | 5.49 | 0.415 | 4.84 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_1_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_1_0_infer.tar) | +| ReXNet_1_3 | 0.7913 | 0.9464 | 3.54 | 4.87 | 6.54 | 0.68 | 7.61 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_1_3_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_1_3_infer.tar) | +| ReXNet_1_5 | 0.8006 | 0.9512 | 3.68 | 5.31 | 7.38 | 0.90 | 9.79 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_1_5_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_1_5_infer.tar) | +| ReXNet_2_0 | 0.8122 | 0.9536 | 4.30 | 6.54 | 9.19 | 1.56 | 16.45 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_2_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_2_0_infer.tar) | +| ReXNet_3_0 | 0.8209 | 0.9612 | 5.74 | 9.49 | 13.62 | 3.44 | 34.83 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ReXNet_3_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ReXNet_3_0_infer.tar) | + + + +## 16. SwinTransformer 系列 [[27](#ref27)] + +关于 SwinTransformer 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[SwinTransformer 系列模型文档](../models/SwinTransformer.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| SwinTransformer_tiny_patch4_window7_224 | 0.8069 | 0.9534 | 6.59 | 9.68 | 16.32 | 4.35 | 28.26 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_tiny_patch4_window7_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_tiny_patch4_window7_224_infer.tar) | +| SwinTransformer_small_patch4_window7_224 | 0.8275 | 0.9613 | 12.54 | 17.07 | 28.08 | 8.51 | 49.56 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_small_patch4_window7_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_small_patch4_window7_224_infer.tar) | +| SwinTransformer_base_patch4_window7_224 | 0.8300 | 0.9626 | 13.37 | 23.53 | 39.11 | 15.13 | 87.70 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window7_224_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window7_224_infer.tar) | +| SwinTransformer_base_patch4_window12_384 | 0.8439 | 0.9693 | 19.52 | 64.56 | 123.30 | 44.45 | 87.70 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window12_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window12_384_infer.tar) | +| SwinTransformer_base_patch4_window7_224[1] | 0.8487 | 0.9746 | 13.53 | 23.46 | 39.13 | 15.13 | 87.70 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window7_224_22kto1k_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window7_224_infer.tar) | +| SwinTransformer_base_patch4_window12_384[1] | 0.8642 | 0.9807 | 19.65 | 64.72 | 123.42 | 44.45 | 87.70 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_base_patch4_window12_384_22kto1k_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_base_patch4_window12_384_infer.tar) | +| SwinTransformer_large_patch4_window7_224[1] | 0.8596 | 0.9783 | 15.74 | 38.57 | 71.49 | 34.02 | 196.43 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_large_patch4_window7_224_22kto1k_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_large_patch4_window7_224_22kto1k_infer.tar) | +| SwinTransformer_large_patch4_window12_384[1] | 0.8719 | 0.9823 | 32.61 | 116.59 | 223.23 | 99.97 | 196.43 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SwinTransformer_large_patch4_window12_384_22kto1k_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SwinTransformer_large_patch4_window12_384_22kto1k_infer.tar) | + +[1]：基于 ImageNet22k 数据集预训练，然后在 ImageNet1k 数据集迁移学习得到。 + + + +## 17. LeViT 系列 [[33](#ref33)] + +关于 LeViT 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[LeViT 系列模型文档](../models/LeViT.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(M) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| LeViT_128S | 0.7598 | 0.9269 | | | | 281 | 7.42 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_128S_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_128S_infer.tar) | +| LeViT_128 | 0.7810 | 0.9371 | | | | 365 | 8.87 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_128_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_128_infer.tar) | +| LeViT_192 | 0.7934 | 0.9446 | | | | 597 | 10.61 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_192_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_192_infer.tar) | +| LeViT_256 | 0.8085 | 0.9497 | | | | 1049 | 18.45 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_256_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_256_infer.tar) | +| LeViT_384 | 0.8191 | 0.9551 | | | | 2234 | 38.45 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/LeViT_384_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/LeViT_384_infer.tar) | + +**注**：与 Reference 的精度差异源于数据预处理不同及未使用蒸馏的 head 作为输出。 + + + +## 18. Twins 系列 [[34](#ref34)] + +关于 Twins 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[Twins 系列模型文档](../models/Twins.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| pcpvt_small | 0.8082 | 0.9552 | 7.32 | 10.51 | 15.27 |3.67 | 24.06 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/pcpvt_small_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/pcpvt_small_infer.tar) | +| pcpvt_base | 0.8242 | 0.9619 | 12.20 | 16.22 | 23.16 | 6.44 | 43.83 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/pcpvt_base_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/pcpvt_base_infer.tar) | +| pcpvt_large | 0.8273 | 0.9650 | 16.47 | 22.90 | 32.73 | 9.50 | 60.99 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/pcpvt_large_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/pcpvt_large_infer.tar) | +| alt_gvt_small | 0.8140 | 0.9546 | 6.94 | 9.01 | 12.27 |2.81 | 24.06 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/alt_gvt_small_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/alt_gvt_small_infer.tar) | +| alt_gvt_base | 0.8294 | 0.9621 | 9.37 | 15.02 | 24.54 | 8.34 | 56.07 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/alt_gvt_base_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/alt_gvt_base_infer.tar) | +| alt_gvt_large | 0.8331 | 0.9642 | 11.76 | 22.08 | 35.12 | 14.81 | 99.27 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/alt_gvt_large_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/alt_gvt_large_infer.tar) | + +**注**：与 Reference 的精度差异源于数据预处理不同。 + + + +## 19. HarDNet 系列 [[37](#ref37)] + +关于 HarDNet 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[HarDNet 系列模型文档](../models/HarDNet.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| HarDNet39_ds | 0.7133 |0.8998 | 1.40 | 2.30 | 3.33 | 0.44 | 3.51 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet39_ds_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet39_ds_infer.tar) | +| HarDNet68_ds |0.7362 | 0.9152 | 2.26 | 3.34 | 5.06 | 0.79 | 4.20 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet68_ds_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet68_ds_infer.tar) | +| HarDNet68| 0.7546 | 0.9265 | 3.58 | 8.53 | 11.58 | 4.26 | 17.58 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet68_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet68_infer.tar) | +| HarDNet85 | 0.7744 | 0.9355 | 6.24 | 14.85 | 20.57 | 9.09 | 36.69 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/HarDNet85_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/HarDNet85_infer.tar) | + + + +## 20. DLA 系列 [[38](#ref38)] + +关于 DLA 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[DLA 系列模型文档](../models/DLA.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| DLA102 | 0.7893 |0.9452 | 4.95 | 8.08 | 12.40 | 7.19 | 33.34 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA102_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA102_infer.tar) | +| DLA102x2 |0.7885 | 0.9445 | 19.58 | 23.97 | 31.37 | 9.34 | 41.42 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA102x2_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA102x2_infer.tar) | +| DLA102x| 0.781 | 0.9400 | 11.12 | 15.60 | 20.37 | 5.89 | 26.40 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA102x_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA102x_infer.tar) | +| DLA169 | 0.7809 | 0.9409 | 7.70 | 12.25 | 18.90 | 11.59 | 53.50 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA169_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA169_infer.tar) | +| DLA34 | 0.7603 | 0.9298 | 1.83 | 3.37 | 5.98 | 3.07 | 15.76 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA34_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA34_infer.tar) | +| DLA46_c |0.6321 | 0.853 | 1.06 | 2.08 | 3.23 | 0.54 | 1.31 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA46_c_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA46_c_infer.tar) | +| DLA60 | 0.7610 | 0.9292 | 2.78 | 5.36 | 8.29 | 4.26 | 22.08 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA60_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA60_infer.tar) | +| DLA60x_c | 0.6645 | 0.8754 | 1.79 | 3.68 | 5.19 | 0.59 | 1.33 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA60x_c_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA60x_c_infer.tar) | +| DLA60x | 0.7753 | 0.9378 | 5.98 | 9.24 | 12.52 | 3.54 | 17.41 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DLA60x_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DLA60x_infer.tar) | + + + +## 21. RedNet 系列 [[39](#ref39)] + +关于 RedNet 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[RedNet 系列模型文档](../models/RedNet.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| RedNet26 | 0.7595 |0.9319 | 4.45 | 15.16 | 29.03 | 1.69 | 9.26 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet26_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet26_infer.tar) | +| RedNet38 |0.7747 | 0.9356 | 6.24 | 21.39 | 41.26 | 2.14 | 12.43 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet38_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet38_infer.tar) | +| RedNet50| 0.7833 | 0.9417 | 8.04 | 27.71 | 53.73 | 2.61 | 15.60 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet50_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet50_infer.tar) | +| RedNet101 | 0.7894 | 0.9436 | 13.07 | 44.12 | 83.28 | 4.59 | 25.76 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet101_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet101_infer.tar) | +| RedNet152 | 0.7917 | 0.9440 | 18.66 | 63.27 | 119.48 | 6.57 | 34.14 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/RedNet152_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/RedNet152_infer.tar) | + + + +## 22. TNT 系列 [[35](#ref35)] + +关于 TNT 系列模型的精度、速度指标如下表所示，更多介绍可以参考：[TNT 系列模型文档](../models/TNT.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +| ---------- | --------- | --------- | ---------------- | ---------------- | -------- | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| TNT_small | 0.8121 |0.9563 | | | 4.83 | 23.68 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/TNT_small_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/TNT_small_infer.tar) | + +**注**：TNT 模型的数据预处理部分 `NormalizeImage` 中的 `mean` 与 `std` 均为 0.5。 + + + +## 23. 其他模型 + +关于 AlexNet [[18](#ref18)]、SqueezeNet 系列 [[19](#ref19)]、VGG 系列 [[20](#ref20)]、DarkNet53 [[21](#ref21)] 等模型的精度、速度指标如下表所示，更多介绍可以参考：[其他模型文档](../models/Others.md)。 + +| 模型 | Top-1 Acc | Top-5 Acc | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | FLOPs(G) | Params(M) | 预训练模型下载地址 | inference模型下载地址 | +|------------------------|-----------|-----------|------------------|------------------|----------|-----------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| +| AlexNet | 0.567 | 0.792 | 0.81 | 1.50 | 2.33 | 0.71 | 61.10 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/AlexNet_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/AlexNet_infer.tar) | +| SqueezeNet1_0 | 0.596 | 0.817 | 0.68 | 1.64 | 2.62 | 0.78 | 1.25 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SqueezeNet1_0_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SqueezeNet1_0_infer.tar) | +| SqueezeNet1_1 | 0.601 | 0.819 | 0.62 | 1.30 | 2.09 | 0.35 | 1.24 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/SqueezeNet1_1_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/SqueezeNet1_1_infer.tar) | +| VGG11 | 0.693 | 0.891 | 1.72 | 4.15 | 7.24 | 7.61 | 132.86 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG11_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG11_infer.tar) | +| VGG13 | 0.700 | 0.894 | 2.02 | 5.28 | 9.54 | 11.31 | 133.05 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG13_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG13_infer.tar) | +| VGG16 | 0.720 | 0.907 | 2.48 | 6.79 | 12.33 | 15.470 | 138.35 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG16_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG16_infer.tar) | +| VGG19 | 0.726 | 0.909 | 2.93 | 8.28 | 15.21 | 19.63 | 143.66 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/VGG19_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/VGG19_infer.tar) | +| DarkNet53 | 0.780 | 0.941 | 2.79 | 6.42 | 10.89 | 9.31 | 41.65 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/DarkNet53_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/DarkNet53_infer.tar) | + + + +## 参考文献 + +[1] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778. + +[2] He T, Zhang Z, Zhang H, et al. Bag of tricks for image classification with convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 558-567. + +[3] Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[C]//Proceedings of the IEEE International Conference on Computer Vision. 2019: 1314-1324. + +[4] Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4510-4520. + +[5] Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017. + +[6] Ma N, Zhang X, Zheng H T, et al. Shufflenet v2: Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 116-131. + +[7] Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1492-1500. + +[8] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7132-7141. + +[9] Gao S, Cheng M M, Zhao K, et al. Res2net: A new multi-scale backbone architecture[J]. IEEE transactions on pattern analysis and machine intelligence, 2019. + +[10] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1-9. + +[11] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-resnet and the impact of residual connections on learning[C]//Thirty-first AAAI conference on artificial intelligence. 2017. + +[12] Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1251-1258. + +[13] Wang J, Sun K, Cheng T, et al. Deep high-resolution representation learning for visual recognition[J]. arXiv preprint arXiv:1908.07919, 2019. + +[14] Chen Y, Li J, Xiao H, et al. Dual path networks[C]//Advances in neural information processing systems. 2017: 4467-4475. + +[15] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708. + +[16] Tan M, Le Q V. Efficientnet: Rethinking model scaling for convolutional neural networks[J]. arXiv preprint arXiv:1905.11946, 2019. + +[17] Mahajan D, Girshick R, Ramanathan V, et al. Exploring the limits of weakly supervised pretraining[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 181-196. + +[18] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105. + +[19] Iandola F N, Han S, Moskewicz M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size[J]. arXiv preprint arXiv:1602.07360, 2016. + +[20] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014. + +[21] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788. + +[22] Ding X, Guo Y, Ding G, et al. Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks[C]//Proceedings of the IEEE International Conference on Computer Vision. 2019: 1911-1920. + +[23] Han K, Wang Y, Tian Q, et al. GhostNet: More features from cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 1580-1589. + +[24] Zhang H, Wu C, Zhang Z, et al. Resnest: Split-attention networks[J]. arXiv preprint arXiv:2004.08955, 2020. + +[25] Radosavovic I, Kosaraju R P, Girshick R, et al. Designing network design spaces[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 10428-10436. + +[26] C.Szegedy, V.Vanhoucke, S.Ioffe, J.Shlens, and Z.Wojna. Rethinking the inception architecture for computer vision. arXiv preprint arXiv:1512.00567, 2015. + +[27] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin and Baining Guo. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. + +[28]Cheng Cui, Tingquan Gao, Shengyu Wei, Yuning Du, Ruoyu Guo, Shuilong Dong, Bin Lu, Ying Zhou, Xueying Lv, Qiwen Liu, Xiaoguang Hu, Dianhai Yu, Yanjun Ma. PP-LCNet: A Lightweight CPU Convolutional Neural Network. + +[29]Mingxing Tan, Quoc V. Le. MixConv: Mixed Depthwise Convolutional Kernels. + +[30]Dongyoon Han, Sangdoo Yun, Byeongho Heo, YoungJoon Yoo. Rethinking Channel Dimensions for Efficient Model Design. + +[31]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby. AN IMAGE IS WORTH 16X16 WORDS: +TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE. + +[32]Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, Herve Jegou. Training data-efficient image transformers & distillation through attention. + +[33]Benjamin Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Herve Jegou, Matthijs Douze. LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference. + +[34]Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, Chunhua Shen. Twins: Revisiting the Design of Spatial Attention in Vision Transformers. + +[35]Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, Yunhe Wang. Transformer in Transformer. + +[36]Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, Jian Sun. RepVGG: Making VGG-style ConvNets Great Again. + +[37]Ping Chao, Chao-Yang Kao, Yu-Shan Ruan, Chien-Hsiang Huang, Youn-Long Lin. HarDNet: A Low Memory Traffic Network. + +[38]Fisher Yu, Dequan Wang, Evan Shelhamer, Trevor Darrell. Deep Layer Aggregation. + +[39]Duo Lim Jie Hu, Changhu Wang, Xiangtai Li, Qi She, Lei Zhu, Tong Zhang, Qifeng Chen. Involution: Inverting the Inherence of Convolution for Visual Recognition. diff --git a/src/PaddleClas/docs/zh_CN/algorithm_introduction/image_classification.md b/src/PaddleClas/docs/zh_CN/algorithm_introduction/image_classification.md new file mode 100644 index 0000000..b55aad0 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/algorithm_introduction/image_classification.md @@ -0,0 +1,78 @@ +# 图像分类任务介绍 + + +## 目录 + + +- [1. 数据集介绍](#1) + - [1.1 ImageNet-1k](#1.1) + - [1.2 CIFAR-10/CIFAR-100](#1.2) +- [2. 图像分类的流程](#2) + - [2.1 数据及其预处理](#2.1) + - [2.2 模型准备](#2.2) + - [2.3 模型训练](#2.3) + - [2.4 模型评估](#2.4) +- [3. 主要算法简介](#3) + +图像分类是根据图像的语义信息将不同类别图像区分开来，是计算机视觉中重要的基本问题，也是图像检测、图像分割、物体跟踪、行为分析等其他高层视觉任务的基础。图像分类在很多领域有广泛应用，包括安防领域的人脸识别和智能视频分析等，交通领域的交通场景识别，互联网领域基于内容的图像检索和相册自动归类，医学领域的图像识别等。 + +一般来说，图像分类通过手工特征或特征学习方法对整个图像进行全部描述，然后使用分类器判别物体类别，因此如何提取图像的特征至关重要。在深度学习算法之前使用较多的是基于词袋(Bag of Words)模型的物体分类方法。而基于深度学习的图像分类方法，可以通过有监督或无监督的方式学习层次化的特征描述，从而取代了手工设计或选择图像特征的工作。深度学习模型中的卷积神经网络(Convolution Neural Network, CNN)近年来在图像领域取得了惊人的成绩，CNN 直接利用图像像素信息作为输入，最大程度上保留了输入图像的所有信息，通过卷积操作进行特征的提取和高层抽象，模型输出直接是图像识别的结果。这种基于"输入-输出"直接端到端的学习方法取得了非常好的效果，得到了广泛的应用。 + +图像分类是计算机视觉里很基础但又重要的一个领域，其研究成果一直影响着计算机视觉甚至深度学习的发展，图像分类有很多子领域，如多标签分类、细粒度分类等，此处只对单标签图像分类做一个简述。 + + +## 1. 数据集介绍 + + +### 1.1 ImageNet-1k + +ImageNet 项目是一个大型视觉数据库，用于视觉目标识别软件研究。该项目已手动注释了 1400 多万张图像，以指出图片中的对象，并在至少 100 万张图像中提供了边框。ImageNet-1k 是 ImageNet 数据集的子集，其包含 1000 个类别。训练集包含 1281167 个图像数据，验证集包含 50000 个图像数据。2010 年以来，ImageNet 项目每年举办一次图像分类竞赛，即 ImageNet 大规模视觉识别挑战赛(ILSVRC)。挑战赛使用的数据集即为 ImageNet-1k。到目前为止，ImageNet-1k 已经成为计算机视觉领域发展的最重要的数据集之一，其促进了整个计算机视觉的发展，很多计算机视觉下游任务的初始化模型都是基于该数据集训练得到的权重。 + + +### 1.2 CIFAR-10/CIFAR-100 + +CIFAR-10 数据集由 10 个类的 60000 个彩色图像组成，图像分辨率为 32x32，每个类有 6000 个图像，其中训练集 5000 张，验证集 1000 张，10 个不同的类代表飞机、汽车、鸟类、猫、鹿、狗、青蛙、马、轮船和卡车。CIFAR-100 数据集是 CIFAR-10 的扩展，由 100 个类的 60000 个彩色图像组成，图像分辨率为 32x32，每个类有 600 个图像，其中训练集 500 张，验证集 100 张。由于这两个数据集规模较小，因此可以让研究人员快速尝试不同的算法。这两个数据集也是图像分类领域测试模型好坏的常用数据集。 + + +## 2. 图像分类的流程 + +将准备好的训练数据做相应的数据预处理后经过图像分类模型，模型的输出与真实标签做交叉熵损失函数，该损失函数描述了模型的收敛方向，遍历所有的图片数据输入模型，对最终损失函数通过某些优化器做相应的梯度下降，将梯度信息回传到模型中，更新模型的权重，如此循环往复遍历多次数据，即可得到一个图像分类的模型。 + + +### 2.1 数据及其预处理 + +数据的质量及数量往往可以决定一个模型的好坏。在图像分类领域，数据包括图像及标签。在大部分情形下，带有标签的数据比较匮乏，所以数量很难达到使模型饱和的程度，为了可以使模型学习更多的图像特征，图像数据在进入模型之前要经过很多图像变换或者数据增强，来保证输入图像数据的多样性，从而保证模型有更好的泛化能力。PaddleClas 提供了训练 ImageNet-1k 的标准图像变换，也提供了 8 中数据增强的方法，相关代码可以[数据处理](../../../ppcls/data/preprocess)，配置文件可以参考[数据增强配置文件](../../../ppcls/configs/ImageNet/DataAugment)。 + + +### 2.2 模型准备 + +在数据确定后，模型往往决定了最终算法精度的上限，在图像分类领域，经典的模型层出不穷，PaddleClas 提供了 36 个系列共 175 个 ImageNet 预训练模型。具体的精度、速度等指标请参考[骨干网络和预训练模型库](./ImageNet_models.md)。 + + +### 2.3 模型训练 + +在准备好数据、模型后，便可以开始迭代模型并更新模型的参数。经过多次迭代最终可以得到训练好的模型来做图像分类任务。图像分类的训练过程需要很多经验，涉及很多超参数的设置，PaddleClas 提供了一些列的[训练调优方法](../models_training/train_strategy.md)，可以快速助你获得高精度的模型。 + + +### 2.4 模型评估 + +当训练得到一个模型之后，如何确定模型的好坏，需要将模型在验证集上进行评估。评估指标一般是 Top1-Acc 或者 Top5-Acc，该指标越高往往代表模型性能越好。 + + +## 3. 主要算法简介 + +- LeNet：Yan LeCun 等人于上个世纪九十年代第一次将卷积神经网络应用到图像分类任务上，创造性的提出了 LeNet，在手写数字识别任务上取得了巨大的成功。 + +- AlexNet：Alex Krizhevsky 等人在 2012 年提出了 AlexNet，并应用于 ImageNet 上，获得了 2012 年 ImageNet 分类比赛的冠军，从此，掀起了深度学习的热潮。 + +- VGG：Simonyan 和 Zisserman 于 2014 年提出了 VGG 网络结构，该网络结构使用了更小的卷积核堆叠整个网络，在 ImageNet 分类上取得了更好的性能，也为之后的网络结构设计提供了新的思路。 + +- GoogLeNet：Christian Szegedy 等人在 2014 年提出了 GoogLeNet，该网络使用了多分支结构和全局平均池化层（GAP），在保持模型精度的同时，模型存储和计算量大大缩减。该网络取得了 2014 年 ImageNet 分类比赛的冠军。 + +- ResNet：Kaiming He 等人在 2015 年提出了 ResNet，通过引入残差模块加深了网络的深度，最终该网络将 ImageNet 分类的识别错误率降低到了 3.6\%，首次超出正常人眼的识别精度。 + +- DenseNet：Huang Gao 等人在 2017 年提出了 DenseNet，该网络设计了一种更稠密连接的 block，在更小的参数量上获得了更高的性能。 + +- EfficientNet：Mingxing Tan 等人在 2019 年提出了 EfficientNet，该网络平衡了网络的宽度、网络的深度、以及输入图片的分辨率，在同样 FLOPS 与参数量的情况下，精度达到了 state-of-the-art 的效果。 + +关于更多的算法介绍，请参考[算法介绍](../models)。 diff --git a/src/PaddleClas/docs/zh_CN/algorithm_introduction/index.rst b/src/PaddleClas/docs/zh_CN/algorithm_introduction/index.rst new file mode 100644 index 0000000..aa56459 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/algorithm_introduction/index.rst @@ -0,0 +1,12 @@ +算法介绍 +================================ + +.. toctree:: + :maxdepth: 2 + + image_classification.md + metric_learning.md + knowledge_distillation.md + model_prune_quantization.md + ImageNet_models.md + DataAugmentation.md diff --git a/src/PaddleClas/docs/zh_CN/algorithm_introduction/knowledge_distillation.md b/src/PaddleClas/docs/zh_CN/algorithm_introduction/knowledge_distillation.md new file mode 100644 index 0000000..5809219 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/algorithm_introduction/knowledge_distillation.md @@ -0,0 +1,86 @@ +# 知识蒸馏 +--- +## 目录 + +* [1. 模型压缩和知识蒸馏方法简介](#1) +* [2. 知识蒸馏应用](#2) +* [3. 知识蒸馏算法介绍](#3) + * [3.1 Response based distillation](#3.1) + * [3.2 Feature based distillation](#3.2) + * [3.3 Relation based distillation](#3.3) +* [4. 参考文献](#4) + +## 1. 模型压缩和知识蒸馏方法简介 + +近年来，深度神经网络在计算机视觉、自然语言处理等领域被验证是一种极其有效的解决问题的方法。通过构建合适的神经网络，加以训练，最终网络模型的性能指标基本上都会超过传统算法。 + +在数据量足够大的情况下，通过合理构建网络模型的方式增加其参数量，可以显著改善模型性能，但是这又带来了模型复杂度急剧提升的问题。大模型在实际场景中使用的成本较高。 + +深度神经网络一般有较多的参数冗余，目前有几种主要的方法对模型进行压缩，减小其参数量。如裁剪、量化、知识蒸馏等，其中知识蒸馏是指使用教师模型(teacher model)去指导学生模型(student model)学习特定任务，保证小模型在参数量不变的情况下，得到比较大的性能提升，甚至获得与大模型相似的精度指标 [1]。 + +目前知识蒸馏的方法大致可以分为以下三种。 + +* Response based distillation：教师模型对学生模型的输出进行监督。 +* Feature based distillation：教师模型对学生模型的中间层 feature map 进行监督。 +* Relation based distillation：对于不同的样本，使用教师模型和学生模型同时计算样本之间 feature map 的相关性，使得学生模型和教师模型得到的相关性矩阵尽可能一致。 + + +## 2. 知识蒸馏应用 + + +知识蒸馏算法在模型轻量化过程任务中应用广泛，对于需要满足特定的精度的任务，通过使用知识蒸馏的方法，我们可以使用更小的模型便能达到要求的精度，从而减小了模型部署的成本。 + +此外，对于相同的模型结构，使用知识蒸馏训练得到的预训练模型精度往往更高，这些预训练模型往往也可以提升下游任务的模型精度。比如在图像分类任务中，基于知识蒸馏算法得到的精度更高的预训练模型，也能够在目标检测、图像分割、OCR、视频分类等任务中获得明显的精度收益。 + + + +## 3. 知识蒸馏算法介绍 + +### 3.1 Response based distillation + +最早的知识蒸馏算法 KD，由 Hinton 提出，训练的损失函数中除了 gt loss 之外，还引入了学生模型与教师模型输出的 KL 散度，最终精度超过单纯使用 gt loss 训练的精度。这里需要注意的是，在训练的时候，需要首先训练得到一个更大的教师模型，来指导学生模型的训练过程。 + +PaddleClas 中提出了一种简单使用的 SSLD 知识蒸馏算法 [6]，在训练的时候去除了对 gt label 的依赖，结合大量无标注数据，最终蒸馏训练得到的预训练模型在 15 个模型上的精度提升平均高达 3%。 + +上述标准的蒸馏方法是通过一个大模型作为教师模型来指导学生模型提升效果，而后来又发展出 DML(Deep Mutual Learning)互学习蒸馏方法 [7]，即通过两个结构相同的模型互相学习。具体的。相比于 KD 等依赖于大的教师模型的知识蒸馏算法，DML 脱离了对大的教师模型的依赖，蒸馏训练的流程更加简单，模型产出效率也要更高一些。 + + +### 3.2 Feature based distillation + +Heo 等人提出了 OverHaul [8], 计算学生模型与教师模型的 feature map distance，作为蒸馏的 loss，在这里使用了学生模型、教师模型的转移，来保证二者的 feature map 可以正常地进行 distance 的计算。 + +基于 feature map distance 的知识蒸馏方法也能够和 `3.1 章节` 中的基于 response 的知识蒸馏算法融合在一起，同时对学生模型的输出结果和中间层 feature map 进行监督。而对于 DML 方法来说，这种融合过程更为简单，因为不需要对学生和教师模型的 feature map 进行转换，便可以完成对齐(alignment)过程。PP-OCRv2 系统中便使用了这种方法，最终大幅提升了 OCR 文字识别模型的精度。 + + +### 3.3 Relation based distillation + + +`3.1` 和 `3.2` 章节中的论文中主要是考虑到学生模型与教师模型的输出或者中间层 feature map，这些知识蒸馏算法只关注个体的输出结果，没有考虑到个体之间的输出关系。 + +Park 等人提出了 RKD [10]，基于关系的知识蒸馏算法，RKD 中进一步考虑个体输出之间的关系，使用 2 种损失函数，二阶的距离损失（distance-wise）和三阶的角度损失（angle-wise） + + +本论文提出的算法关系知识蒸馏（RKD）迁移教师模型得到的输出结果间的结构化关系给学生模型，不同于之前的只关注个体输出结果，RKD 算法使用两种损失函数：二阶的距离损失(distance-wise)和三阶的角度损失(angle-wise)。在最终计算蒸馏损失函数的时候，同时考虑 KD loss 和 RKD loss。最终精度优于单独使用 KD loss 蒸馏得到的模型精度。 + + +## 4. 参考文献 + +[1] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015. + +[2] Bagherinezhad H, Horton M, Rastegari M, et al. Label refinery: Improving imagenet classification through label progression[J]. arXiv preprint arXiv:1805.02641, 2018. + +[3] Yalniz I Z, Jégou H, Chen K, et al. Billion-scale semi-supervised learning for image classification[J]. arXiv preprint arXiv:1905.00546, 2019. + +[4] Cubuk E D, Zoph B, Mane D, et al. Autoaugment: Learning augmentation strategies from data[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2019: 113-123. + +[5] Touvron H, Vedaldi A, Douze M, et al. Fixing the train-test resolution discrepancy[C]//Advances in Neural Information Processing Systems. 2019: 8250-8260. + +[6] Cui C, Guo R, Du Y, et al. Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones[J]. arXiv preprint arXiv:2103.05959, 2021. + +[7] Zhang Y, Xiang T, Hospedales T M, et al. Deep mutual learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4320-4328. + +[8] Heo B, Kim J, Yun S, et al. A comprehensive overhaul of feature distillation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 1921-1930. + +[9] Du Y, Li C, Guo R, et al. PP-OCRv2: Bag of Tricks for Ultra Lightweight OCR System[J]. arXiv preprint arXiv:2109.03144, 2021. + +[10] Park W, Kim D, Lu Y, et al. Relational knowledge distillation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 3967-3976. diff --git a/src/PaddleClas/docs/zh_CN/algorithm_introduction/metric_learning.md b/src/PaddleClas/docs/zh_CN/algorithm_introduction/metric_learning.md new file mode 100644 index 0000000..07cbda8 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/algorithm_introduction/metric_learning.md @@ -0,0 +1,36 @@ +# Metric Learning +---- +## 目录 + +* [1. 简介](#1) +* [2. 应用](#2) +* [3. 算法](#3) + * [3.1 Classification based](#3.1) + * [3.2 Pairwise based](#3.2) + + +## 1. 简介 + 在机器学习中，我们经常会遇到度量数据间距离的问题。一般来说，对于可度量的数据，我们可以直接通过欧式距离(Euclidean Distance)，向量内积(Inner Product)或者是余弦相似度(Cosine Similarity)来进行计算。但对于非结构化数据来说，我们却很难进行这样的操作，如计算一段视频和一首音乐的匹配程度。由于数据格式的不同，我们难以直接进行上述的向量运算，但先验知识告诉我们 ED(laugh_video, laugh_music) < ED(laugh_video, blue_music), 如何去有效得表征这种”距离”关系呢? 这就是 Metric Learning 所要研究的课题。 + + Metric learning 全称是 Distance Metric Learning，它是通过机器学习的形式，根据训练数据，自动构造出一种基于特定任务的度量函数。Metric Learning 的目标是学习一个变换函数（线性非线性均可）L，将数据点从原始的向量空间映射到一个新的向量空间，在新的向量空间里相似点的距离更近，非相似点的距离更远，使得度量更符合任务的要求，如下图所示。 Deep Metric Learning，就是用深度神经网络来拟合这个变换函数。 +![example](../../images/ml_illustration.jpg) + + +## 2. 应用 + Metric Learning 技术在生活实际中应用广泛，如我们耳熟能详的人脸识别(Face Recognition)、行人重识别(Person ReID)、图像检索(Image Retrieval)、细粒度分类(Fine-grained classification)等。随着深度学习在工业实践中越来越广泛的应用，目前大家研究的方向基本都偏向于 Deep Metric Learning(DML). + + 一般来说, DML 包含三个部分: 特征提取网络来 map embedding, 一个采样策略来将一个 mini-batch 里的样本组合成很多个 sub-set, 最后 loss function 在每个 sub-set 上计算 loss. 如下图所示： + ![image](../../images/ml_pipeline.jpg) + + +## 3. 算法 + Metric Learning 主要有如下两种学习范式： + +### 3.1 Classification based: + 这是一类基于分类标签的 Metric Learning 方法。这类方法通过将每个样本分类到正确的类别中，来学习有效的特征表示，学习过程中需要每个样本的显式标签参与 Loss 计算。常见的算法有 [L2-Softmax](https://arxiv.org/abs/1703.09507), [Large-margin Softmax](https://arxiv.org/abs/1612.02295), [Angular Softmax](https://arxiv.org/pdf/1704.08063.pdf), [NormFace](https://arxiv.org/abs/1704.06369), [AM-Softmax](https://arxiv.org/abs/1801.05599), [CosFace](https://arxiv.org/abs/1801.09414), [ArcFace](https://arxiv.org/abs/1801.07698)等。 + 这类方法也被称作是 proxy-based, 因为其本质上优化的是样本和一堆 proxies 之间的相似度。 + +### 3.2 Pairwise based: + 这是一类基于样本对的学习范式。他以样本对作为输入，通过直接学习样本对之间的相似度来得到有效的特征表示，常见的算法包括：[Contrastive loss](http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf), [Triplet loss](https://arxiv.org/abs/1503.03832), [Lifted-Structure loss](https://arxiv.org/abs/1511.06452), [N-pair loss](https://papers.nips.cc/paper/2016/file/6b180037abbebea991d8b1232f8a8ca9-Paper.pdf), [Multi-Similarity loss](https://arxiv.org/pdf/1904.06627.pdf)等 + +2020 年发表的[CircleLoss](https://arxiv.org/abs/2002.10857)，从一个全新的视角统一了两种学习范式，让研究人员和从业者对 Metric Learning 问题有了更进一步的思考。 diff --git a/src/PaddleClas/docs/zh_CN/algorithm_introduction/model_prune_quantization.md b/src/PaddleClas/docs/zh_CN/algorithm_introduction/model_prune_quantization.md new file mode 100644 index 0000000..b47cd17 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/algorithm_introduction/model_prune_quantization.md @@ -0,0 +1,54 @@ +# 模型裁剪、量化算法介绍 + +深度学习因其计算复杂度或参数冗余，在一些场景和设备上限制了相应的模型部署，需要借助模型压缩、优化加速、异构计算等方法突破瓶颈。模型压缩算法能够有效降低参数冗余，从而减少存储占用、通信带宽和计算复杂度，有助于深度学习的应用部署。其中模型量化、裁剪应用比较广泛。在 PaddleClas 中，主要应该应用以下两种算法。 + +- 量化方法：PACT 量化 +- 裁剪：FPGM 裁剪 + +其中具体算法参数请参考 [PaddeSlim](https://github.com/PaddlePaddle/PaddleSlim/)。 + +## 目录 + +* [1. PACT 量化方法](#1) +* [2. FPGM 裁剪](#2) + + + +## 1. PACT 量化方法 + +模型量化主要包括两个部分，一是对权重 Weight 量化，一是针对激活值 Activation 量化。同时对两部分进行量化，才能获得最大的计算效率收益。权重可以借助网络正则化等手段，让权重分布尽量紧凑，减少离群点、不均匀分布情况发生，而对于激活值还缺乏有效的手段。 + +**PACT 量化（PArameterized Clipping acTivation**是一种新的量化方法，该方法通过在量化激活值之前去掉一些离群点，将模型量化带来的精度损失降到最低，甚至比原模型准确率更高。提出方法的背景是作者发现：“在运用权重量化方案来量化 activation 时，激活值的量化结果和全精度结果相差较大”。作者发现，activation 的量化可能引起的误差很大(相较于 weight 基本在 0 到 1 范围内，activation 的值的范围是无限大的，这是 RELU 的结果)，所以提出**截断式 RELU** 的激活函数。该截断的上界，即$α$ 是可学习的参数，这保证了每层能够通过训练学习到不一样的量化范围，最大程度降低量化带来的舍入误差。其中量化的示意图如下图所示，**PACT** 解决问题的方法是，不断裁剪激活值范围，使得激活值分布收窄，从而降低量化映射损失。**PACT** 通过对激活数值做裁剪，从而减少激活分布中的离群点，使量化模型能够得到一个更合理的量化 scale，降低量化损失。 + +![](../../images/algorithm_introduction/quantization.jpg) + +**PACT** 量化公式如下： + +![](../../images/algorithm_introduction/quantization_formula.png) + + +可以看出 PACT 思想是用上述量化代替 *ReLU* 函数，对大于零的部分进行一个截断操作，截断阈值为$a$。但是在*PaddleSlim*中对上述公式做了进一步的改进，其改进如下： + + +![](../../images/algorithm_introduction/quantization_formula_slim.png) + +经过如上改进后，在激活值和待量化的 OP（卷积，全连接等）之间插入 *PACT* 预处理，不只对大于 0 的分布进行截断，同时也对小于 0 的部分做同样的限制，从而更好地得到待量化的范围，降低量化损失。同时，截断阈值是一个可训练的参数，在量化训练过程中，模型会自动的找到一个合理的截断阈值，从而进一步降低量化精度损失。 + +算法具体参数请参考 PaddleSlim 中[参数介绍](https://github.com/PaddlePaddle/PaddleSlim/blob/release/2.0.0/docs/zh_cn/api_cn/dygraph/quanter/qat.rst#qat)。 + + + +## 2. FPGM 裁剪 + +模型剪枝是减小模型大小，提升预测效率的一种非常重要的手段。在之前的网络剪枝文章中一般将网络 filter 的范数作为其重要性度量，**范数值较小的代表的 filter 越不重要**，将其从网络中裁剪掉，反之也就越重要。而**FPGM**认为之前的方法要依赖如下两点 + +- filter 的范数偏差应该比较大，这样重要和非重要的 filter 才可以很好区分开 +- 不重要的 filter 的范数应该足够的小 + +基于此，**FPGM**利用层中 filter 的几何中心特性，由于那些靠近中心的 filter 可以被其它的表达，因而可以将其剔除，从而避免了上面提到的两点剪枝条件，从信息的冗余度出发，而不是选择范数少的进行剪枝。下图展示了 **FPGM** 方法与之前方法的不同，具体细节请详看[论文](https://openaccess.thecvf.com/content_CVPR_2019/papers/He_Filter_Pruning_via_Geometric_Median_for_Deep_Convolutional_Neural_Networks_CVPR_2019_paper.pdf)。 + +![](../../images/algorithm_introduction/fpgm.png) + + + +算法具体参数请参考 PaddleSlim 中[参数介绍](https://github.com/PaddlePaddle/PaddleSlim/blob/release/2.0.0/docs/zh_cn/api_cn/dygraph/pruners/fpgm_filter_pruner.rst#fpgmfilterpruner)。 diff --git a/src/PaddleClas/docs/zh_CN/conf.py b/src/PaddleClas/docs/zh_CN/conf.py new file mode 100644 index 0000000..889ef59 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/conf.py @@ -0,0 +1,69 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) +import sphinx_rtd_theme +from recommonmark.parser import CommonMarkParser +#import sphinx-markdown-tables +# -- Project information ----------------------------------------------------- + +project = 'PaddleClas' +copyright = '2021, PaddleClas' +author = 'PaddleClas' + +# The full version, including alpha/beta/rc tags +release = '2.3.0' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +source_parsers = { + '.md': CommonMarkParser, +} +source_suffix = ['.rst', '.md'] +extensions = [ + 'recommonmark', + 'sphinx_markdown_tables' + ] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = 'zh_CN' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = "sphinx_rtd_theme" +html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] diff --git a/src/PaddleClas/docs/zh_CN/data_preparation/classification_dataset.md b/src/PaddleClas/docs/zh_CN/data_preparation/classification_dataset.md new file mode 100644 index 0000000..66f7180 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/data_preparation/classification_dataset.md @@ -0,0 +1,123 @@ +# 图像分类任务数据集说明 + +本文档将介绍 PaddleClas 所使用的图像分类任务数据集格式，以及图像分类领域的常见数据集介绍。 + +--- + +## 目录 + +- [1. 数据集格式说明](#1) +- [2. 图像分类任务常见数据集介绍](#2) + - [2.1 ImageNet1k](#2.1) + - [2.2 Flowers102](#2.2) + - [2.3 CIFAR10 / CIFAR100](#2.3) + - [2.4 MNIST](#2.4) + - [2.5 NUS-WIDE](#2.5) + + +## 1. 数据集格式说明 + +PaddleClas 使用 `txt` 格式文件指定训练集和测试集，以 `ImageNet1k` 数据集为例，其中 `train_list.txt` 和 `val_list.txt` 的格式形如： + +```shell +# 每一行采用"空格"分隔图像路径与标注 + +# 下面是 train_list.txt 中的格式样例 +train/n01440764/n01440764_10026.JPEG 0 +... + +# 下面是 val_list.txt 中的格式样例 +val/ILSVRC2012_val_00000001.JPEG 65 +... +``` + +## 2. 图像分类任务常见数据集介绍 + +这里整理了常用的图像分类任务数据集，持续更新中，欢迎各位小伙伴补充完善～ + + +### 2.1 ImageNet1k + +[ImageNet](https://image-net.org/)项目是一个大型视觉数据库，用于视觉目标识别研究任务，该项目已手动标注了 1400 多万张图像。ImageNet-1k 是 ImageNet 数据集的子集，其包含 1000 个类别。训练集包含 1281167 个图像数据，验证集包含 50000 个图像数据。2010 年以来，ImageNet 项目每年举办一次图像分类竞赛，即 ImageNet 大规模视觉识别挑战赛（ILSVRC）。挑战赛使用的数据集即为 ImageNet-1k。到目前为止，ImageNet-1k 已经成为计算机视觉领域发展的最重要的数据集之一，其促进了整个计算机视觉的发展，很多计算机视觉下游任务的初始化模型都是基于该数据集训练得到的。 + +数据集 | 训练集大小 | 测试集大小 | 类别数 | 备注| +:------:|:---------------:|:---------------------:|:-----------:|:-----------: +[ImageNet1k](http://www.image-net.org/challenges/LSVRC/2012/)|1.2M| 50k | 1000 | + +从官方下载数据后，按如下格式组织数据，即可在 PaddleClas 中使用 ImageNet1k 数据集进行训练。 + +```bash +PaddleClas/dataset/ILSVRC2012/ +|_ train/ +| |_ n01440764 +| | |_ n01440764_10026.JPEG +| | |_ ... +| |_ ... +| | +| |_ n15075141 +| |_ ... +| |_ n15075141_9993.JPEG +|_ val/ +| |_ ILSVRC2012_val_00000001.JPEG +| |_ ... +| |_ ILSVRC2012_val_00050000.JPEG +|_ train_list.txt +|_ val_list.txt +``` + + +### 2.2 Flowers102 + +数据集 | 训练集大小 | 测试集大小 | 类别数 | 备注| +:------:|:---------------:|:---------------------:|:-----------:|:-----------: +[flowers102](https://www.robots.ox.ac.uk/~vgg/data/flowers/102/)|1k | 6k | 102 | + +将下载的数据解压后，可以看到以下目录 + +```shell +jpg/ +setid.mat +imagelabels.mat +``` + +将以上文件放置在 `PaddleClas/dataset/flowers102/` 下 + +通过运行 `generate_flowers102_list.py` 生成 `train_list.txt` 和 `val_list.txt`： + +```shell +python generate_flowers102_list.py jpg train > train_list.txt +python generate_flowers102_list.py jpg valid > val_list.txt +``` + +按照如下结构组织数据： + +```shell +PaddleClas/dataset/flowers102/ +|_ jpg/ +| |_ image_03601.jpg +| |_ ... +| |_ image_02355.jpg +|_ train_list.txt +|_ val_list.txt +``` + + +### 2.3 CIFAR10 / CIFAR100 + +CIFAR-10 数据集由 10 个类的 60000 个彩色图像组成，图像分辨率为 32x32，每个类有 6000 个图像，其中训练集 5000 张，验证集 1000 张，10 个不同的类代表飞机、汽车、鸟类、猫、鹿、狗、青蛙、马、轮船和卡车。CIFAR-100 数据集是 CIFAR-10 的扩展，由 100 个类的 60000 个彩色图像组成，图像分辨率为 32x32，每个类有 600 个图像，其中训练集 500 张，验证集 100 张。 + +数据集地址：http://www.cs.toronto.edu/~kriz/cifar.html + + +### 2.4 MNIST + +MMNIST 是一个非常有名的手写体数字识别数据集，在很多资料中，这个数据集都会被用作深度学习的入门样例。其包含 60000 张图片数据，50000 张作为训练集，10000 张作为验证集，每张图片的大小为 28 * 28。 + +数据集地址：http://yann.lecun.com/exdb/mnist/ + + +### 2.5 NUS-WIDE + +NUS-WIDE 是一个多分类数据集。该数据集包含 269648 张图片, 81 个类别，每张图片被标记为该 81 个类别中的某一类或某几类。 + +数据集地址：https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html diff --git a/src/PaddleClas/docs/zh_CN/data_preparation/index.rst b/src/PaddleClas/docs/zh_CN/data_preparation/index.rst new file mode 100644 index 0000000..c851ca5 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/data_preparation/index.rst @@ -0,0 +1,8 @@ +数据准备 +================================ + +.. toctree:: + :maxdepth: 2 + + recognition_dataset.md + classification_dataset.md diff --git a/src/PaddleClas/docs/zh_CN/data_preparation/recognition_dataset.md b/src/PaddleClas/docs/zh_CN/data_preparation/recognition_dataset.md new file mode 100644 index 0000000..2fb1ce4 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/data_preparation/recognition_dataset.md @@ -0,0 +1,132 @@ +# 图像识别任务数据集说明 + +本文档将介绍 PaddleClas 所使用的图像识别任务数据集格式，以及图像识别领域的常见数据集介绍。 + +--- + + +## 目录 + +- [1. 数据集格式说明](#1) +- [2. 图像识别任务常见数据集介绍](#2) + - [2.1 通用图像识别数据集](#2.1) + - [2.2 垂类图像识别数据集](#2.2) + - [2.2.1 动漫人物识别](#2.2.1) + - [2.2.2 商品识别](#2.2.2) + - [2.2.3 Logo 识别](#2.2.3) + - [2.2.4 车辆识别](#2.2.4) + + + +## 1. 数据集格式说明 + +与分类任务数据集不同，图像检索任务的数据集分为以下三部分： + +* 训练集合（train dataset）：用来训练模型，使模型能够学习该集合的图像特征。 +* 底库数据集合（gallery dataset）：用来提供图像检索任务中的底库数据，该集合可与训练集或测试集相同，也可以不同，当与训练集相同时，测试集的类别体系应与训练集的类别体系相同。 +* 测试数据集合（query dataset）：用来测试模型的好坏，通常要对测试集的每一张测试图片进行特征提取，之后和底库数据的特征进行距离匹配，得到识别结果，后根据识别结果计算整个测试集的指标。 + +训练集、底库数据集和测试数据集均使用 `txt` 文件指定，以 `CUB_200_2011` 数据集为例，训练数据集 `train_list.txt` 文件内容格式如下所示： + +```shell +# 采用"空格"作为分隔符号 +... +train/99/Ovenbird_0136_92859.jpg 99 2 +... +train/99/Ovenbird_0128_93366.jpg 99 6 +... +``` + +验证数据集(`CUB_200_2011` 中既是 gallery dataset，也是 query dataset)`test_list.txt` 文件内容格式如下所示： + +```shell +# 采用"空格"作为分隔符号 +... +test/200/Common_Yellowthroat_0126_190407.jpg 200 1 +... +test/200/Common_Yellowthroat_0114_190501.jpg 200 6 +... +``` + +每行数据使用“空格”分割，三列数据的含义分别是训练数据的路径、训练数据的 label 信息、训练数据的 unique id。 + +**注意**： + +1.当 gallery dataset 和 query dataset 相同时，为了去掉检索得到的第一个数据（检索图片本身无须评估），每个数据需要对应一个 unique id（每张图片的 id 不同即可，可以用行号来表示 unique id），用于后续评测 mAP、recall@1 等指标。yaml 配置文件的数据集选用 `VeriWild`。 + +2.当 gallery dataset 和 query dataset 不同时，无需增加 unique id，`query_list.txt` 和 `gallery_list.txt` 均为两列，分别是训练数据的路径、训练数据的 label 信息。yaml 配置文件的数据集选用 `ImageNetDataset`。 + + +## 2. 图像识别任务常见数据集介绍 + +这里整理了常用的图像识别任务数据集，持续更新中，欢迎各位小伙伴补充完善～ + + +### 2.1 通用图像识别数据集 +- SOP: SOP 数据集是通用识别研究领域、MetricLearning 技术研究方向常用的一个商品数据集, 其包含从 eBay.com 下载的 22,634 个产品的 120,053 张图片。其中, 训练集包含图片 59551 张, 类别数 11318; 验证集包含图片 60502 张，类别数 11316 个。 + + 地址: https://cvgl.stanford.edu/projects/lifted_struct/ + +- Cars196: +Cars 数据集包含了 196 类汽车的 16185 张图像。数据被分成 8144 张训练图像和 8041 张测试图像，每个类大致以 50-50 的比例分割。级别通常是在制造，模型，年，例如 2012 特斯拉模型 S 或 2012 宝马 M3 双门跑车。 + + 地址: https://ai.stanford.edu/~jkrause/cars/car_dataset.html +- CUB_200_2011: CUB_200_2011 数据集是由加州理工学院在 2010 年提出的细粒度数据集，也是目前细粒度分类识别研究的基准图像数据集。该数据集共有 11788 张鸟类图像，包含 200 类鸟类子类，其中训练数据集有 5994 张图像，测试集有 5794 张图像，每张图像均提供了图像类标记信息，图像中鸟的 bounding box，鸟的关键 part 信息，以及鸟类的属性信息，数据集如下图所示。 + + 地址: http://www.vision.caltech.edu/visipedia/CUB-200-2011.html + +- In-shop Clothes： In-shop Clothes 是 DeepFashion 数据集的 4 个子集之一, 它是一个卖家秀图片集，每个商品 id，有多张不同角度的卖家秀，放在同一个文件夹内。该数据集共包含 7982 件商品，共 52712 张图像，每张图片都有 463 中属性，Bbox，landmarks，以及店铺描述。 + + 地址： http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html + + +### 2.2 垂类图像识别数据集 + + +#### 2.2.1 动漫人物识别 ++ iCartoonFace: iCartoonFace 是由爱奇艺开放的目前全球最大的手工标注卡通人物检测数据集与识别数据集，它包含超过 5013 个卡通人物、389678 张高质量实景图片。相比于其他数据集，它具有大规模、高质量、多样性丰富、挑战难度大等特点，是目前研究动漫人物识别最常用的数据集之一。 + + 地址： http://challenge.ai.iqiyi.com/detail?raceId=5def69ace9fcf68aef76a75d + ++ Manga109： Manga109 是 2020.5 月发布的一个用于研究卡通人物检测和识别的数据集，其包含 21142 张图片，官方不允许用于商用。该数据集旗下的子集 Manga109-s，可以供工业使用，主要用于文本检测、基于线稿的任务检索、角色图像生成等任务。 + + 地址：http://www.manga109.org/en/ + ++ IIT-CFW：IIF-CFW 数据集共包含 8928 个带有标注的明星人物卡通头像，覆盖 100 个人物形象，每个人卡通头像数不等。 另外，其还提供了 1000 张真实人脸照（100 个公众人物，每个人 10 张真实头像）。该数据集既可以用于研究动漫人物识别，也经常被用于研究跨模态的检索任务。 + + 地址： http://cvit.iiit.ac.in/research/projects/cvit-projects/cartoonfaces + + +#### 2.2.2 商品识别 ++ AliProduct: AliProduct 数据集是目前开源最大的商品数据集，它是一个 SKU 级别的图像分类数据集，包含 5 万类别、300 万张商品图像，商品图像的类别和总量均为业界之最。此数据集中涵盖了大量的生活用品、食物等，数据集中没有人工标注，数据较脏，数据分布较不均衡，且有很多相似的商品图片。 + + 地址: https://retailvisionworkshop.github.io/recognition_challenge_2020/ + ++ Product-10k: Products-10k 数据集中的所有图片均来自京东商城。数据集中共包含 1 万个经常购买的 SKU。所有 SKU 组织成一个层次结构。总共有近 19 万张图片。在实际应用场景中，图像量的分布是不均衡的。所有图像都由生产专家团队手工检查/标记。 + + 地址：https://www.kaggle.com/c/products-10k/data?select=train.csv + ++ DeepFashion-Inshop: 同通用图像识别数据集中的 In-shop Clothes + + +### 2.2.3 Logo 识别 ++ Logo-2K+： Logo-2K+是一个仅用于 logo 图像识别的数据集，其包含 10 个大类，2341 个小类和 167140 张图片。 + + 地址： https://github.com/msn199959/Logo-2k-plus-Dataset + ++ Tsinghua-Tencent 100K： 该数据集是从 10 万张腾讯街景全景图中创建的一个大型交通标志基准数据集。它提供包含 30000 个交通标志实例的 100000 张图像。这些图像涵盖了照度和天气条件的巨大变化。基准测试中的每个交通标志都标注了类别标签、边界框和像素掩码。 它总共包含 222 个类别(0 background + 221 traffic signs) + + 地址： https://cg.cs.tsinghua.edu.cn/traffic-sign/ + +### 2.2.4 车辆识别 ++ CompCars： 图像主要来自网络和监控数据，其中网络数据包含 163 个汽车制造商、1716 个汽车型号的汽车。共 136,726 张全车图像，27,618 张部分车图像。其中网络汽车数据包含 bounding box、视角、5 个属性（最大速度、排量、车门数、车座数、汽车类型）。监控数据包含 50,000 张前视角图像。 + + 地址： http://mmlab.ie.cuhk.edu.hk/datasets/comp_cars/ + ++ BoxCars： 此数据集共包含 21250 辆车、63750 张图像、27 个汽车制造商、148 个细类别，此数据集全部来自监控数据。 + + 地址： https://github.com/JakubSochor/BoxCars + ++ PKU-VD Dataset：该数据集包含了两个大型车辆数据集（VD1 和 VD2），它们分别从两个城市的真实世界不受限制的场景拍摄图像。其中 VD1 是从高分辨率交通摄像头获得的，VD2 中的图像则是从监视视频中获取的。作者对原始数据执行车辆检测，以确保每个图像仅包含一辆车辆。由于隐私保护的限制，所有车牌号码都已被黑色覆盖遮挡。所有车辆图像均从前视图进行拍摄。 数据集中为每个图像提供了多样化的属性注释，包括身份编号，精确的车辆模型和车辆颜色。VD1 原先包含 1097649 张图像，1232 种车俩模型，11 种车辆颜色，但删除图像里面有多辆车辆以及从车辆后方拍摄的图片，该数据集仅剩 846358 张图像，141756 辆车辆。 VD2 包含 807260 张图像，79763 辆车辆，1112 种车辆模型，11 种车辆颜色。 + + 地址： https://pkuml.org/resources/pku-vds.html diff --git a/src/PaddleClas/docs/zh_CN/faq_series/faq_2020_s1.md b/src/PaddleClas/docs/zh_CN/faq_series/faq_2020_s1.md new file mode 100644 index 0000000..e780a54 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/faq_series/faq_2020_s1.md @@ -0,0 +1,349 @@ +# 图像分类常见问题汇总 - 2020 第 1 季 + + +## 目录 +* [1. 第 1 期](#1)(2020.11.03) +* [2. 第 2 期](#2)(2020.11.11) +* [3. 第 3 期](#3)(2020.11.18) +* [4. 第 4 期](#4)(2020.12.07) +* [5. 第 5 期](#5)(2020.12.17) +* [6. 第 6 期](#6)(2020.12.30) + + +## 第 1 期 + +### Q1.1: PaddleClas 可以用来做什么? +**A**：PaddleClas 是飞桨为工业界和学术界所准备的一个图像分类任务的工具集，助力使用者训练出更好的视觉模型和应用落地。PaddleClas 提供了基于图像分类的模型训练、评估、预测、部署全流程的服务，方便大家更加高效地学习图像分类。具体地，PaddleClas 中包含如下一些特性。 + +* PaddleClas 提供了 36 个系列的分类网络结构(ResNet, ResNet_vd, MobileNetV3, Res2Net, HRNet 等)和训练配置，175 个预训练模型和性能评估与预测，供大家选择并使用。 +* PaddleClas 提供了 TensorRT 预测、python inference、c++ inference、Paddle-Lite 预测部署、PaddleServing、PaddleHub 等多种预测部署推理方案，在方便在多种环境中进行部署推理。 +* PaddleClas 提供了一种简单的 SSLD 知识蒸馏方案，基于该方案蒸馏模型的识别准确率普遍提升 3% 以上。 +* PaddleClas 支持 AutoAugment、Cutout、Cutmix 等 8 种数据增广算法详细介绍、代码复现和在统一实验环境下的效果评估。 +* PaddleClas 支持在 Windows/Linux/MacOS 环境中基于 CPU/GPU 进行使用。 + +### Q1.2: ResNet 系列模型是什么？有哪些模型？为什么在服务器端如此推荐 ResNet 系列模型？ +**A**: ResNet 中创新性地引入了残差结构，通过堆叠多个残差结构从而构建了 ResNet 网络。实验表明使用残差块可以有效地提升收敛速度和精度，PaddleClas 中，ResNet 从小到达，依次有包含 18、34、50、101、152、200 层的 ResNet 结构，ResNet 系列模型于 2015 年被提出，在不同的应用场景中，如分类、检测、分割等，都已经验证过其有效性，业界也早已对其进行了大量优化，该系列模型在速度和精度方面都有着非常明显的优势，对基于 TensorRT 以及 FP16 的预测支持得也很好，因而推荐大家使用 ResNet 系列模型；由于其模型所占存储相对较大，因此常用于服务器端。更多关于 ResNet 模型的介绍可以参考论文 [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385)。 + +### Q1.3: ResNet_vd 和 ResNet、ResNet_vc 结构有什么区别呢？ +**A**: +ResNet_va 至 vd 的结构如下图所示，ResNet 最早提出时为 va 结构，在降采样残差模块这个部分，在左边的特征变换通路中(Path A)，第一个 1x1 卷积部分就行了降采样，从而导致信息丢失（卷积的 kernel size 为 1，stride 为 2，输入特征图中 有部分特征没有参与卷积的计算）；在 vb 结构中，把降采样的步骤从最开始的第一个 1x1 卷积调整到中间的 3x3 卷积中，从而避免了信息丢失的问题，PaddleClas 中的 ResNet 模型默认就是 ResNet_vb；vc 结构则是将最开始这个 7x7 的卷积变成 3 个 3x3 的卷积，在感受野不变的情况下，计算量和存储大小几乎不变，而且实验证明精度相对于 vb 结构有所提升；vd 结构是修改了降采样残差模块右边的特征通路(Path B)。把降采样的过程由平均池化这个操作去替代了，这一系列的改进(va->vd)，几乎没有带来新增的预测耗时，结合适当的训练策略，比如说标签平滑以及 mixup 数据增广，精度可以提升高达 2.7%。 + +![](../../images/faq/ResNet_vabcd_structure.png) +### Q1.4 如果确定使用 ResNet 系列模型，怎么根据实际的场景需求选用不同的模型呢？ +**A**: + +ResNet 系列模型中，相比于其他模型，ResNet_vd 模型在预测速度几乎不变的情况下，精度有非常明显的提升，因此推荐大家使用 ResNet_vd 系列模型。 +[ResNet 及其 vd 系列模型文档](../models/ResNet_and_vd.md)中给出了 batch size=4 的情况下，在 T4 GPU 上，不同模型的的预测耗时、FLOPs、Params 与精度的变化曲线，可以根据自己自己的实际部署场景中的需求，去选择合适的模型，如果希望模型存储大小尽可能小或者预测速度尽可能快，则可以使用 ResNet18_vd 模型，如果希望获得尽可能高的精度，则建议使用 ResNet152_vd 或者 ResNet200_vd 模型。更多关于 ResNet 系列模型的介绍可以参考文档：[ResNet 及其 vd 系列模型文档](../models/ResNet_and_vd.md)。 + +* 精度-预测速度变化曲线 + + ![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.png) +* 精度-params 变化曲线 + + ![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.params.png) +* 精度-flops 变化曲线 + + ![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.flops.png) +### Q1.5 在网络中的 block 里 conv-bn-relu 是固定的形式吗？ + +**A**: 在 batch-norm 出现之前，主流的卷积神经网络的固定形式是 conv-relu。在现阶段的卷积神经网络中，conv-bn-relu 是大部分网络中 block 的固定形式，这样的设计是相对鲁棒的结构，此外，DenseNet 中的 block 选用的是 bn-relu-conv 的形式，ResNet-V2 中也使用的是这种组合方式。在 MobileNetV2 中，为了不丢失信息，部分 block 中间的层没有使用 relu 激活函数，选用的是 conv-bn 的形式。 + +### Q1.6 ResNet34 与 ResNet50 的区别？ + +**A**: ResNet 系列中有两种不同的 block，分别是 basic-block 和 bottleneck-block，堆叠较多这样的 block 组成了 ResNet 网络。basic-block 是带有 shortcut 的两个 3x3 的卷积核的堆叠，bottleneck-block 是带有 shortcut 的 1x1 卷积核、3x3 卷积核、1x1 卷积核的堆叠，所以 basic-block 中有两层，bottleneck-block 有三层。ResNet34 和 ResNet50 中堆叠的 block 数相同，但是堆叠的种类分别是 basic-block 和 bottleneck-block。 + +### Q1.7 大卷积核一定可以带来正向收益吗？ + +**A**: 不一定，将网络中的所有卷积核都增大未必会带来性能的提升，甚至会有有损性能，在论文 [MixConv: Mixed Depthwise Convolutional Kernels](https://arxiv.org/abs/1907.09595) +中指出，在一定范围内提升卷积核大小对精度的提升有正向作用，但是超出后会有损精度。所以考虑到模型的大小、计算量等问题，一般不选用大的卷积核去设计网络。同时，在 [PP-LCNet](../models/PP-LCNet.md) 文章中，也有关于大卷积核的实验。 + + +## 第 2 期 + +### Q2.1: PaddleClas 如何训练自己的 backbone？ + +**A**：具体流程如下: +* 首先在 ppcls/arch/backbone/model_zoo/ 文件夹下新建一个自己的模型结构文件，即你自己的 backbone，模型搭建可以参考 resnet.py; +* 然后在 ppcls/arch/backbone/\_\_init\_\_.py 中添加自己设计的 backbone 的类; +* 其次配置训练的 yaml 文件，此处可以参考 ppcls/configs/ImageNet/ResNet/ResNet50.yaml; +* 最后启动训练即可。 + + +### Q2.2: 如何利用已有的模型和权重对自己的分类任务进行迁移？ + +**A**: 具体流程如下: +* 首先，好的预训练模型往往会有更好的迁移效果，所以建议选用精度较高的预训练模型，PaddleClas 提供了一系列业界领先的预训练模型，建议使用； +* 其次，要根据迁移的数据集的规模来确定训练超参数，一般超参数需要调试才可以寻找到一个局部最优值，如果没有相关经验，建议先从 learning rate 开始调起，一般来说，规模较小的数据集使用较小的 learning rate，如 0.001，另外，建议学习率使用 warmup 策略，避免过大的学习率破坏预训练模型的权重。在迁移过程中，也可以设置 backbone 中不同层的学习率，往往从网络的头部到尾补学习率逐渐减小效果较好。在数据集规模较小的时候，也可以使用数据增强策略，PaddleClas 提供了 8 中强有力的数据增强策略，为更高的精度保驾护航。 +* 训练结束后，可以反复迭代上述过程，直到寻找到局部最优值。 + +### Q2.3: PaddleClas 中 configs 下的默认参数适合任何一个数据集吗？ + +**A**: PaddleClas 中的 ppcls/configs/ImageNet/下的配置文件默认参数是 ImageNet-1k 的训练参数，这个参数并不适合所有的数据集，具体数据集需要在此基础上进一步调试。 + + +### Q2.4 PaddleClas 中的不同的模型使用了不同的分辨率，标配的应该是多少呢？ + +**A**: PaddleClas 严格遵循了论文作者的使用的分辨率。自 2012 年 AlexNet 以来，大多数的卷积神经网络在 ImageNet 上训练的分辨率为 224x224，Google 在设计 InceptionV3 的时候为了适应网络结构将分辨率调至 299x299，之后其推出的 Xception、InceptionV4 也是使用的该分辨率。此外，在 EfficeintNet 中，作者分析了不同规模的网络应该使用不同的分辨率，所以该系列网络中每个不同大小的网络都使用了不同的分辨率。在实际使用场景中，推荐使用默认的分辨率，当然，层数较深或者宽度较大的网络也可以尝试使用更大的分辨率。 + + +### Q2.5 PaddleClas 中提供了很多 ssld 模型，其应用的价值是？ + +**A**: PaddleClas 中提供了很多 ssld 预训练模型，其通过半监督知识蒸馏的方法获得了更好的预训练权重，在迁移任务或者下游视觉任务中，无须替换结构文件、只需要替换精度更高的 ssld 预训练模型即可提升精度，如在 PaddleSeg 中，[HRNet](https://github.com/PaddlePaddle/PaddleSeg/blob/release/v0.7.0/docs/model_zoo.md) 使用了 ssld 预训练模型的权重后，精度大幅度超越业界同样的模型的精度，在 PaddleDetection 中，[PP-YOLO](https://github.com/PaddlePaddle/PaddleDetection/blob/release/0.4/configs/ppyolo/README_cn.md) 使用了 ssld 预训练权重后，在较高的 baseline 上仍有进一步的提升。使用 ssld 预训练权重做分类的迁移表现也很抢眼，在 [SSLD 蒸馏策略](../advanced_tutorials/knowledge_distillation.md) 部分介绍了知识蒸馏对于分类任务迁移的收益。 + + + +## 第 3 期 + +### Q3.1: DenseNet 模型相比于 ResNet 有什么改进呢？有哪些特点或者应用场景呢？ + +**A**: DenseNet 相比于 ResNet，设计了一个更激进的密集连接机制，通过考虑特征重用和旁路的设置，进一步减少了参数量，而且从一定程度上缓解了梯度弥散的问题，因为引入了更加密集的连接，因此模型更容易训练，而且具有一定的正则化效果。在数据量不是很多的图像分类场景中，DenseNet 是一个不错的选择。更多关于 DenseNet 的介绍与系列模型可以参考 [DenseNet 模型文档](../models/DPN_DenseNet.md)。 + + + +### Q3.2: DPN 网络相比于 DenseNet 有哪些改进呢？ + +**A**：DPN 的全称是 Dual Path Networks，即双通道网络。该网络是由 DenseNet 和 ResNeXt 结合的一个网络，其证明了 DenseNet 能从靠前的层级中提取到新的特征，而 ResNeXt 本质上是对之前层级中已提取特征的复用。作者进一步分析发现，ResNeXt 对特征有高复用率，但冗余度低，DenseNet 能创造新特征，但冗余度高。结合二者结构的优势，作者设计了 DPN 网络。最终 DPN 网络在同样 FLOPS 和参数量下，取得了比 ResNeXt 与 DenseNet 更好的结果。更多关于 DPN 的介绍与系列模型可以参考 [DPN 模型文档](../models/DPN_DenseNet.md)。 + + +### Q3.3: 怎么使用多个模型进行预测融合呢？ + +**A** 使用多个模型进行预测的时候，建议首先将预训练模型导出为 inference 模型，这样可以摆脱对网络结构定义的依赖，可以参考[模型导出脚本](../../../tools/export_model.py)进行模型导出，之后再参考 [inference 模型预测脚本](../../../deploy/python/predict_cls.py)进行预测即可，在这里需要根据自己使用模型的数量创建多个 predictor。 + + +### Q3.4: PaddleClas 中怎么增加自己的数据增广方法呢？ + +**A**： + +* 对于单张图像的增广，可以参考[基于单张图片的数据增广脚本](../../../ppcls/data/preprocess/ops)，参考 `ResizeImage `或者 `CropImage` 等数据算子的写法，创建一个新的类，然后在 `__call__` 中，实现对应的增广方法即可。 +* 对于一个 batch 图像的增广，可以参考[基于 batch 数据的数据增广脚本](../../../ppcls/data/preprocess/batch_ops)，参考 `MixupOperator` 或者 `CutmixOperator` 等数据算子的写法，创建一个新的类，然后在 `__call__` 中，实现对应的增广方法即可。 + + +### Q3.5: 怎么进一步加速模型训练过程呢？ + +**A**： + +* 可以使用自动混合精度进行训练，这在精度几乎无损的情况下，可以有比较明显的速度收益，以 ResNet50 为例，PaddleClas 中使用自动混合精度训练的配置文件可以参考：[ResNet50_fp16.yml](../../../ppcls/configs/ImageNet/ResNet/ResNet50_fp16.yaml)，主要就是需要在标准的配置文件中添加以下几行 + +``` +# mixed precision training +AMP: + scale_loss: 128.0 + use_dynamic_loss_scaling: True + use_pure_fp16: &use_pure_fp16 True +``` + +* 可以开启 dali，将数据预处理方法放在 GPU 上运行，在模型比较小时（reader 耗时占比更高一些），开启 dali 会带来比较明显的训练速度收益，在训练的时候，添加 `-o Global.use_dali=True` 即可使用 dali 进行训练，更多关于 dali 安装与介绍可以参考：[dali 安装教程](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html#nightly-builds)。 + + +## 第 4 期 + +### Q4.1: PaddlePaddle 的模型文件都有哪几种？ + +**A**: +* PaddlePaddle 保存的模型相关文件有两类： +* 一类是用于*推理部署*的文件，包括后缀名为 “`pdiparams`”、“`model`” 的文件，其中 “`pdiparams`” 文件存储了模型参数信息，“`model`” 文件存储了模型网络结构信息，对于推理部署文件，使用 `paddle.jit.save` 与 `paddle.jit.load` 接口进行保存、加载。 +* 另一类模型相关文件则是用于*训练调优*过程中，包括后缀名为 “`pdparams`” 和 “`pdopt`” 的文件，其中 “`pdparams`” 文件存储了训练过程中的模型参数信息，“`pdopt`” 文件存储了模型训练过程中的优化器信息，对于训练调优文件，使用 `paddle.save` 与 `paddle.load` 接口进行保存、加载。 +* 利用推理部署文件，即可构建模型网络结构并加载模型参数，用于预测，利用训练调优文件，即可加载模型参数、优化器信息，用于恢复训练过程。 + +### Q4.2: HRNet 的创新点体现在哪里？ + +**A**: +* 在图像分类领域，大部分神经网络的设计思想是提取图像的高维特征，具体来说，通常输入图像的空间分辨率较高，通过多层卷积、池化，可以逐步得到空间分辨率更低，但是维度更高的特征图，然后可用于分类等场景。 +* 然而 *HRNet* 的作者认为这种逐步降低空间分辨率的设计思想并不适合目标检测（图像区域层次的分类任务）、语义分割（图像像素层次的分类任务）等场景，因为空间分辨率在逐步降低的过程中，会丢失很多信息，最终学习得到的特征难以表达原始图像在高空间分辨率的信息，而区域层次分类任务和像素层次分类任务都对空间精度十分敏感。 +* 因此 *HRNet* 的作者提出了并联不同空间分辨率特征图的思想，与此相对，*VGG* 等神经网络则是通过不同的卷积池化层来串联不同空间分辨率的特征图。并且，*HRNet* 通过连接同等深度、不同空间分辨率的特征图，使得不同空间分辨率特征图的信息可以得到充分交换，具体的网络结构如下图所示。 + +![](../../images/faq/HRNet.png) + +### Q4.3: 在 HRNet 中，对于不同空间分辨率的特征图之间，是如何建立连接的？ + +**A**: +* 首先，在 *HRNet* 中，对特征图使用 *stride* 为 *2* 的 *3 × 3* 卷积，可以得到低空间分辨率但是为度更高的特征图；而对低空间分辨率特征图先使用 *1 × 1* 卷积进行通道数匹配，再使用最近邻插值的方式进行上采样，即可得到与高空间分辨率特征图相同空间分辨率、通道数的特征图；而对于相同空间分辨率的特征图，直接进行恒等映射即可。具体如下图所示。 + +![](../../images/faq/HRNet_block.png) + +### Q4.4: 模型中的“SE”表示什么意思？ + +**A**: +* SE 表示该模型使用了 SE 结构。SE 结构来自于 2017 年 ImageNet 分类比赛的冠军方案 *Squeeze-and-Excitation Networks（SENet）*，*SENet* 提出的 SE 结构可以迁移到任何其他网络中。其创新点是通过额外学习 *scale* 向量作为权重作用到特征图上, *scale* 向量维度与特征图通道数相同，学习到的 *scale* 向量中每个维度上的数值表示对该维度特征通道的增强或减弱的大小，以此达到对重要的特征通道进行增强，不重要特征通道减弱的效果，从而让提取的特征指向性更强。 + +### Q4.5: SE 结构具体如何实现的？ + +![](../../images/faq/SE_structure.png) +**A**: +* *SE*结构具体如上图所示，首先，*Ftr* 表示常规的卷积操作，*X* 和 *U* 则是 *Ftr* 的输入与输出的特征图，在得到特征图*U*后，使用 *Fsq* 和 *Fex* 操作求得 *scale* 向量，*scale* 向量维度为 *C*，与 *U* 通道数相同，因此可以通过乘积的方式作用到 *U* 上，进而得到 *X~*。 +* 具体地，*Fsq* 为 *Global Average Pooling* 操作，*SENet* 作者将其称之为 *Squeeze*，因为该操作可以将 *U* 从 *C × H × W* 压缩到 *C × 1 × 1*，对 *Fsq* 的输出再做 *Fex* 操作。 +* *Fex*操作表示两次全连接，作者将该操作称为 *Excitation*。其中第一次全连接将向量的维度从 *1 × 1 × C* 压缩到 *1 × 1 × C/r*，然后使用 *RELU*，再通过第二次全连接将向量的维度恢复到 *C*，这样操作的目的是为了减小计算量，*SENet* 作者通过实验得出结论：在 *r=16* 时可以获得增益与计算量之间的平衡。 +* 对于*Fsq*部分，关键是求得 *C* 维的向量，因此不局限于使用 *Global Average Pooling* 操作，*SENet* 作者认为，最终求得的 *scale* 是按通道分别作用于 *U* 的，因此需要基于对应通道的信息计算对应的 *scale*，故使用了最简单的 *Global Average Pooling* 操作，最终求得的 *scale* 向量表示了不同通道之间的分布关系，而忽略了同一个通道中的分布关系。 +* 对于 *Fex* 部分，其作用是为了在每一个 *mini batch* 上的训练来求得基于所有训练数据的分布。因为我们的训练是在*mini batch*上进行的，而基于全部训练数据求得的 *scale* 才是最佳的，使用 *Fex* 部分，可以通过在每个 *mini batch* 上的训练来求得更为逼近全部训练数据的 *scale*。 + + +## 第 5 期 + +### Q5.1 如何选择优化器？ + +**A**:自深度学习发展以来，就有很多关于优化器的研究者工作，优化器的目的是为了让损失函数尽可能的小，从而找到合适的权重来完成某项任务。目前业界主要用到的优化器有 SGD、RMSProp、Adam、AdaDelt 等，其中由于带 momentum 的 SGD 优化器广泛应用于学术界和工业界(此处仅限于分类任务)，所以我们发布的模型也大都使用该优化器来实现损失函数的梯度下降。带 momentum 的 SGD 优化器有两个劣势，其一是收敛速度慢，其二是初始学习率的设置需要依靠大量的经验，然而如果初始学习率设置得当并且迭代轮数充足，该优化器也会在众多的优化器中脱颖而出，使得其在验证集上获得更高的准确率。一些自适应学习率的优化器如 Adam、RMSProp 等，收敛速度往往比较快，但是最终的收敛精度会稍差一些。如果追求更快的收敛速度，我们推荐使用这些自适应学习率的优化器，如果追求更高的收敛精度，我们推荐使用带 momentum 的 SGD 优化器。具体到数据集来说： + +- ImageNet-1k: 建议只使用带 momentum 的 SGD 优化器。 + +- 其他数据集（默认加载 ImageNet-1k 预训练）: 加载预训练模型的时候可以考虑使用 Adam 等优化器（效果可能会更好），但使用带 momentum 的 SGD 优化器是绝对是比较不错的方案。 + +另外，为了进一步加速训练，Lookahead 优化器也是一个不错的选择，在 ImageNet-1k 上，其可以保证在更快的收敛速度下拥有相同的收敛精度，但在部分数据集上表现不太稳定，需要进一步调参。 + +### Q5.2 如何设置初始学习率以及学习率下降策略？ + +**A**:学习率的选择往往和优化器以及数据和任务有关系。学习率决定了网络种权重更新的速度。学习率越低，损失函数的变化速度就越慢。虽然使用低学习率可以确保不会错过任何局部极小值，但也意味着将花费更长的时间来进行收敛，特别是在被困在高原区域的情况下。 + +在整个训练过程中，我们不能使用同样的学习率来更新权重，否则无法到达最优点，所以需要在训练过程中调整学习率的大小。在训练初始阶段，由于权重处于随机初始化的状态，损失函数下降较快，所以可以设置一个较大的学习率。在训练后期，由于权重已经接近最优值，较大的学习率无法进一步寻找最优值，所以需要设置一个较小的学习率。至于学习率下降策略，很多研究者或者从业人员使用的学习率下降方式是 piecewise_decay(step_decay)，即阶梯式下降学习率，此外，很多研究者也提出了学习率的其他下降方式，如 polynomial_decay（多项式下降）、exponential_decay（指数下降）,cosine_decay（余弦下降）等，其中 cosine_decay 无需调整超参数，鲁棒性也比较高，所以成为现在提高模型精度首选的学习率下降方式。 + +Cosine_decay 和 piecewise_decay 的学习率变化曲线如下图所示，容易观察到，在整个训练过程中，cosine_decay 都保持着较大的学习率，所以其收敛较为缓慢，但是最终的收敛效果较 peicewise_decay 更好一些。 + +![](../../images/models/lr_decay.jpeg) + +另外，从图中我们也可以看到，cosine_decay 中只有少数轮数使用了较小的学习率，这样会影响到最终的精度，所以为了使得 cosine_decay 发挥更好的效果，建议迭代更多的轮数。 + +最后，如果使用较大的 batch_size 训练神经网络时，建议您使用 warmup 策略。Warmup 策略顾名思义就是让学习率先预热一下，在训练初期不直接使用最大的学习率，而是用一个逐渐增大的学习率去训练网络，当学习率增大到最高点时，再去衰减学习率的值。实验表明，在 batch_size 较大时，warmup 可以稳定提升模型的精度。具体到数据集来说： + +- ImageNet-1k:建议 batch-size 大小为 256、初始学习率为 0.1，cosine-decay 下降学习率。 + +- 其他数据集（默认加载 ImageNet-1k 预训练）: 数据集规模越大，初始学习率也越大，但最好不要超过 0.1（batch-size 为 256 时候），数据集规模越小，初始学习率也越小，当数据集较小时，使用 warmup 也会带来一定的精度提升，学习率下降策略仍旧推荐 cosine-decay。 + +### Q5.3 如何设置 batch-size 的大小？ + +**A**:Batch_size 是训练神经网络中的一个重要的超参数，该值决定了一次将多少数据送入神经网络中训练。之前有研究者通过实验发现，当 batch_size 的值与学习率的值呈线性关系时，收敛精度几乎不受影响。在训练 ImageNet-1k 数据时，大部分的神经网络选择的初始学习率为 0.1，batch_size 是 256。具体到数据集来说： + +- ImageNet-1k: 学习率设置为 0.1\*k,batch_size 设置为 256\*k。 + +- 其他数据集（默认加载 ImageNet-1k 预训练）: 可以根据实际情况设置（如更小的学习率），但在调整学习率或者 batch-size 时，要同时调整另外一个值。 + +### Q5.4 weight_decay 是什么？怎么设置？ + +**A**:过拟合是机器学习中常见的一个名词，简单理解即为模型在训练数据上表现很好，但在测试数据上表现较差，在图像分类问题中，同样存在过拟合的问题，为了避免过拟合，很多正则方式被提出，其中，weight_decay 是其中一个广泛使用的避免过拟合的方式。当使用 SGD 优化器时，weight_decay 等价于在最终的损失函数后添加 L2 正则化，L2 正则化使得网络的权重倾向于选择更小的值，最终整个网络中的参数值更趋向于 0，模型的泛化性能相应提高。在各大深度学习框架的实现中，该值表达的含义是 L2 正则前的系数，在飞桨框架中，该值的名称是 L2Decay，所以以下都称其为 L2Decay。该系数越大，表示加入的正则越强，模型越趋于欠拟合状态。具体到数据集来说： + +- ImageNet-1k：大多数的网络将该参数值设置为 1e-4，在一些小的网络如 MobileNet 系列网络中，为了避免网络欠拟合，该值设置为 1e-5~4e-5 之间。下表展示了 MobileNetV1_x0_25 在 ImageNet-1k 上使用不同 L2Decay 的精度情况。由于 MobileNetV1_x0_25 是一个比较小的网络，所以 L2Decay 过大会使网络趋向于欠拟合状态，所以在该网络中，相对 1e-4，3e-5 是更好的选择。 + +| 模型 | L2Decay | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV1_x0_25 | 1e-4 | 43.79%/67.61% | 50.41%/74.70% | +| MobileNetV1_x0_25 | 3e-5 | 47.38%/70.83% | 51.45%/75.45% | + +另外，该值的设置也和训练过程中是否使用其他正则化有关系。如果训练过程中的数据预处理比较复杂，相当于训练任务变的更难，可以将该值适当减小，下表展示了在 ImageNet-1k 上，ResNet50 在使用 RandAugment 预处理方式后使用不同 L2Decay 的精度。容易观察到，在任务变难后，使用更小的 l2_decay 有助于模型精度的提升。 + +| 模型 | L2Decay | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| ResNet50 | 1e-4 | 75.13%/90.42% | 77.65%/93.79% | +| ResNet50 | 7e-5 | 75.56%/90.55% | 78.04%/93.74% | + +- 其他数据集（默认加载 ImageNet-1k 预训练）：在做迁移任务的时候，最好不要改变训练 ImageNet-1k 时的 L2Decay 的值（即训练得到预训练时的 L2Decay 值，每个 backbone 对应的 L2Decay 值都在相应的训练 yaml 配置文件中），一般的数据集只改变学习率足够。 + + +### Q5.5 是否使用 label_smoothing，如何设置其中的参数值？ + +**A**:Label_smoothing 是深度学习中的一种正则化方法，其全称是 Label Smoothing Regularization(LSR)，即标签平滑正则化。在传统的分类任务计算损失函数时，是将真实的 one hot 标签与神经网络的输出做相应的交叉熵计算，而 label_smoothing 是将真实的 one hot 标签做一个标签平滑的处理，使得网络学习的标签不再是一个 hard label，而是一个有概率值的 soft label，其中在类别对应的位置的概率最大，其他位置概率是一个非常小的数。在 label_smoothing 中，epsilon 参数描述了将标签软化的程度，该值越大，经过 label smoothing 后的标签向量的标签概率值越小，标签越平滑，反之，标签越趋向于 hard label。具体到数据集来说： + +- ImageNet-1k：在训练 ImageNet-1k 的实验里通常将该值设置为 0.1，ResNet50 大小级别及其以上的模型在使用 label_smooting 后，精度有稳定的提升。下表展示了 ResNet50_vd 在使用 label_smoothing 前后的精度指标。 + +| 模型 | Use_label_smoothing(0.1) | Test acc1 | +|:--:|:--:|:--:| +| ResNet50_vd | 0 | 77.9% | +| ResNet50_vd | 1 | 78.4% | + +同时，由于 label_smoohing 相当于一种正则方式，在相对较小的模型上，精度提升不明显甚至会有所下降，下表展示了 ResNet18 在 ImageNet-1k 上使用 label_smoothing 前后的精度指标。可以明显看到，在使用 label_smoothing 后，精度有所下降。 + +| 模型 | Use_label_smoohing(0.1) | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| ResNet18 | 0 | 69.81%/87.70% | 70.98%/89.92% | +| ResNet18 | 1 | 68.00%/86.56% | 70.81%/89.89% | + +如何在较小的模型中也可以让 label-smoothing 有效，这里有一个技巧，即在 Global-Average-Pool 后接一个 1000-2000 大小的全连接层，该技巧可以与 label-smoothing 同时作用，发挥更好的效果。 + +- 其他数据集（默认加载 ImageNet-1k 预训练）：使用 label-smooth 之后往往都会提升精度，规模越小的数据集 epsilon 值可以越大，在一些规模较小的细粒度图像中，最佳模型通常是在该值设置到 0.4-0.5 时获得的。 + +### Q5.6 默认的图像预处理中 random-crop 还可以调整吗？怎么调整？ + +**A**:在 ImageNet-1k 数据的标准预处理中，random_crop 函数中定义了 scale 和 ratio 两个值，两个值分别确定了图片 crop 的大小和图片的拉伸程度，其中 scale 的默认取值范围是 0.08-1(lower_scale-upper_scale),ratio 的默认取值范围是 3/4-4/3(lower_ratio-upper_ratio)。在非常小的网络训练中，此类数据增强会使得网络欠拟合，导致精度有所下降。为了提升网络的精度，可以使其数据增强变的更弱，即增大图片的 crop 区域或者减弱图片的拉伸变换程度。可以分别通过增大 lower_scale 的值或缩小 lower_ratio 与 upper_scale 的差距来实现更弱的图片变换。具体到数据集来说： + +- ImageNet-1k：不是特别小的网络建议只用默认值，特别小的网络可以调大 lower_scale 的值（增大 crop 区域面积）或者缩小 ratio 值的范围（减弱图像伸缩变换），特别大的网络可以调小 lower_scale 的值（减小 crop 面积）或者增大 ratio 值的范围（增强图像伸缩变换）。下表列出了使用不同 lower_scale 训练 MobileNetV2_x0_25 的精度，可以看到，增大图片的 crop 区域面积后训练精度和验证精度均有提升。 + +| 模型 | Scale 取值范围 | Train_acc1/acc5 | Test_acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV2_x0_25 | [0.08,1] | 50.36%/72.98% | 52.35%/75.65% | +| MobileNetV2_x0_25 | [0.2,1] | 54.39%/77.08% | 53.18%/76.14% | + +- 其他数据集（默认加载 ImageNet-1k 预训练）：建议使用默认值，如果过拟合较严重，可以考虑调小 lower_scale 的值（减小 crop 面积）或者增大 ratio 值的范围（增强图像伸缩变换）。 + +### Q5.7 目前常用数据增广有哪些？如何选择？ + +**A**:一般来说，数据集的规模对性能影响至关重要，但是图片的标注往往比较昂贵，所以有标注的图片数量往往比较稀少，在这种情况下，数据的增广尤为重要。在训练 ImageNet-1k 的标准数据增广中，主要使用了 Random_Crop 与 Random_Flip 两种数据增广方式，然而，近些年，越来越多的数据增广方式被提出，如 cutout、mixup、cutmix、AutoAugment 等。实验表明，这些数据的增广方式可以有效提升模型的精度。具体到数据集来说： + +- ImageNet-1k：下表列出了 ResNet50 在 8 种不同的数据增广方式的表现，可以看出，相比 baseline，所有的数据增广方式均有收益，其中 cutmix 是目前最有效的数据增广。更多数据增广的介绍请参考[**数据增广章节**](../advanced_tutorials/DataAugmentation.md)。 + +| 模型 | 数据增广方式 | Test top-1 | +|:--:|:--:|:--:| +| ResNet50 | 标准变换 | 77.31% | +| ResNet50 | Auto-Augment | 77.95% | +| ResNet50 | Mixup | 78.28% | +| ResNet50 | Cutmix | 78.39% | +| ResNet50 | Cutout | 78.01% | +| ResNet50 | Gridmask | 77.85% | +| ResNet50 | Random-Augment | 77.70% | +| ResNet50 | Random-Erasing | 77.91% | +| ResNet50 | Hide-and-Seek | 77.43% | + +- 其他数据集（默认加载 ImageNet-1k 预训练）：在其他数据集中除了使用 Auto-Augment，一般都会有精度的提升，Auto-Augment 会针对每一个数据集搜索的独立超参数，该超参数决定了数据如何处理，所以默认的 ImageNet-1k 的超参数并不适合所有的数据集，当然您可以使用 Random-Augment 来替代 Auto-Augment。其他策略可以正常使用，对于比较难的任务或者比较小的网络，建议不要使用较强的数据增广。 + +此外，多种数据增广也可以叠加使用，当数据集较为简单或数据规模较小时，叠加数据增广可以进一步提升精度。 + +### Q5.8 如何通过 train_acc 和 test_acc 确定调优策略？ + +**A**:在训练网络的过程中，通常会打印每一个 epoch 的训练集准确率和验证集准确率，二者刻画了该模型在两个数据集上的表现。通常来说，训练集的准确率反映了经过 Random-Crop 后的数据的精度，由于数据经过 Random-Crop 后，数据往往较难，所以训练集的准确率和验证集的准确率往往不是一个概念。 + +- ImageNet-1k：通常来说，训练集准确率比验证集准确率微高或者二者相当是比较不错的状态。如果发现训练集的准确率比验证集高很多，说明在这个任务上已经过拟合，需要在训练过程中加入更多的正则，如增大 L2Decay 的值，加入更多的数据增广策略，加入 label_smoothing 策略等；如果发现训练集的准确率比验证集低一些，说明在这个任务上可能欠拟合，需要在训练过程中减弱正则效果，如减小 L2Decay 的值，减少数据增广方式，增大图片 crop 区域面积，减弱图片拉伸变换，去除 label_smoothing 等。 + +- 其他数据集（默认加载 ImageNet-1k 预训练）：基本和训练 ImageNet-1k 的调整策略相当，此外，在其他数据集上如果模型趋向于过拟合（train acc 远大于 test acc）状态，也可以使用更优的预训练权重，PaddleClas 为常用的网络提供了 SSLD 的蒸馏预训练权重，其比 ImageNet-1k 的权重更优，您可以优先选择。 + +- **【备注】** 不太建议根据 loss 来重新调整训练策略，在使用不同的数据增广后，train loss 的大小差异较大，如使用 Cutmix 或者 RandAugment 后，train loss 会大于 test loss，当数据增广策略减弱后，train loss 会小于 test loss，所以较难调整。 + +### Q5.9 如何通过预训练模型提升自己的数据集的精度？ + +**A**:在现阶段图像识别领域中，加载预训练模型来训练自己的任务已成为普遍的做法，相比从随机初始化开始训练，加载预训练模型往往可以提升特定任务的精度。一般来说，业界广泛使用的预训练模型是通过训练 128 万张图片 1000 类的 ImageNet-1k 数据集得到的，该预训练模型的 fc 层权重是是一个 k\*1000 的矩阵，其中 k 是 fc 层以前的神经元数，在加载预训练权重时，无需加载 fc 层的权重。在学习率方面，如果您的任务训练的数据集特别小（如小于 1 千张），我们建议你使用较小的初始学习率，如 0.001（batch_size:256,下同），以免较大的学习率破坏预训练权重。如果您的训练数据集规模相对较大（大于 10 万），我们建议你尝试更大的初始学习率，如 0.01 或者更大。如果目标数据集较小，也可以冻结一些浅层的权重。此外，如果训练一个特定垂类的小数据集，也可以先在相关的大的数据集上训练一个预训练权重，再在该权重上用较小的学习率微调模型。 + +### Q5.10 现有的策略已经让模型的精度趋于饱和，如何进一步提升特定模型的精度？ + +**A**:如果现有的策略不能进一步提升模型的精度，说明在现有数据集和现有的策略下，模型几乎到达饱和状态，这里提供两种进一步提升模型精度的方法。 + +- 挖掘相关数据：用在现有数据集上训练饱和的模型去对相关的数据做预测，将置信度较高的数据打 label 后加入训练集进一步训练，如此循环操作，可进一步提升模型的精度。 + +- 知识蒸馏：可以先使用一个较大的模型在该数据集上训练一个精度较高的 teacher model，然后使用该 teacher model 去教导一个 Student model，其中，Student model 即为目标模型。PaddleClas 提供了百度自研的 SSLD 知识蒸馏方案，即使在 ImageNet-1k 这么有挑战的分类任务上，其也能稳定提升 3% 以上。SSLD 知识蒸馏的的章节请参考 [**SSLD 知识蒸馏**](../advanced_tutorials/knowledge_distillation.md)。 + + + +## 第 6 期 + +### Q6.1: PaddleClas 的几个分支有什么区别？应该如何选择？ + +**A**: PaddleClas 目前共有 3 种分支： + +* 开发分支：develop 分支是 PaddleClas 的开发分支，也是更新最快的分支。所有的新功能、新改动都会先在 develop 分支上进行。如果想追踪 PaddleClas 的最新进展，可以关注这个分支。这个分支主要支持动态图，会跟着 paddlepaddle 的版本一起更新。 + +* 稳定版本分支（如 release/2.1.3）：快速更新能够让关注者了解最新进展，但也会带来不稳定性。因此在一些关键的时间点，我们会从 develop 分支中拉出分支，提供稳定的版本，最新的稳定版分支也是默认分支。需要注意，无特殊情况，我们只会维护最新的 release 稳定分支，并且一般只会修复 bug，而不更新新的特性和模型。 + +* 静态图分支（static）：static 分支是使用静态图版本的分支，主要用来支持一些老用户的使用，也只进行一些简单维护，不会更新新的特性和模型。不建议新用户使用静态图分支。老用户如果有条件，也建议迁到动态图分支或稳定版本分支。 + +总的来说，如果想跟进 PaddleClas 的最新进展，建议选择 develop 分支，如果需要稳定版本，建议选择最新的稳定版本分支。 + +### Q6.2: 什么是静态图模式？ + +**A**: 静态图模式即为声明式编程模式。许多深度学习框架如 tensorflow，mxnet 等最初都使用这种模式。在静态图模式中，需要先定义好模型结构，之后框架会根据模型结构进行编译和优化，构建"计算图"。可以简单的理解为，静态图模式是"计算图"静态不变的模式。静态图的优势在于编译器一般只需要构建一次计算图，效率相对较高，缺点在于不够灵活，调试麻烦。例如在 paddle 中运行一次静态图模型，需要完整所有的运算，之后根据特定的 key 来提取输出，无法实时得到结果。 + +### Q6.3: 什么是动态图模式？ + +**A**: 动态图模式即为命令式编程模式，用户无需预先定义网络结构，每行代码都可以直接运行得到结果。相比静态图模式，动态图模式对用户更加友好，调试也更方便。此外，动态图模式的结构设计也更加灵活，可以在运行过程中随时调整结构。 + +PaddleClas 目前持续更新的 develop 分支和稳定版本的 release 分支，主要采用动态图模式。如果您是新用户，建议使用动态图模式来进行开发和训练。如果推理预测时有性能需求，可以在训练完成后，将动态图模型转为静态图模型提高效率。 + +### Q6.5: 构建分类数据集时，如何构建"背景"类别的数据？ + +**A**: 实际使用中，常常需要自己构建一个分类数据集来进行训练。除了所需要的类别数据之外，还需要一个额外的类别，即"背景"类别。例如做一个猫狗分类，猫为一类，狗为一类，如果我们的分类器只有两类，那么输入一张兔子的图片，也会被强制的分到这两个类别中的一个。因此在训练时，应添加一些非目标类别的数据，作为"背景"类别的数据。 + +构建"背景"类别的数据时，首先应从实际需求的角度出发。还是以猫狗分类器为例，如果实际测试的数据都是动物，那么"背景"类别的数据就应该包含一些除猫狗之外的动物。而如果测试的数据还包含更多类别，例如一棵树，那么"背景"类别的数据就要设置的更加丰富。 +简单来说，"背景"类别的数据，要根据实际场景中可能出现的情况去收集。情况越多，需要涵盖的数据种类就越多，任务也会相应的越困难。因此实际处理中，最好能对问题进行限制，避免浪费资源和算力。 diff --git a/src/PaddleClas/docs/zh_CN/faq_series/faq_2021_s1.md b/src/PaddleClas/docs/zh_CN/faq_series/faq_2021_s1.md new file mode 100644 index 0000000..cc60bf5 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/faq_series/faq_2021_s1.md @@ -0,0 +1,251 @@ +# 图像分类常见问题汇总 - 2021 第 1 季 + + +## 目录 +* [1. 第 1 期](#1)(2021.01.05) +* [2. 第 2 期](#2)(2021.01.14) +* [3. 第 3 期](#3)(2020.01.21) +* [4. 第 4 期](#4)(2021.01.28) +* [5. 第 5 期](#5)(2021.02.03) + + +## 第 1 期 + +### Q1.1: 在模型导出时，发现导出的 inference model 预测精度很低，这块是为什么呢？ + +**A**：可以从以下几个方面排查 + +* 需要先排查下预训练模型路径是否正确。 +* 模型导出时，默认的类别数为 1000，如果预训练模型是自定义的类别数，则在导出的时候需要指定参数 `--class_num=k`，k 是自定义的类别数。 +* 可以对比下 `tools/infer/infer.py` 和 `tools/infer/predict.py` 针对相同输入的输出 class id 与 score，如果完全相同，则可能是预训练模型自身的精度很差。 + +### Q1.2: 训练样本的类别不均衡，这个该怎么处理呢？ + +**A**：有以下几种比较常用的处理方法。 + +* 从采样的角度出发的话 + * 可以对样本根据类别进行动态采样，每个类别都设置不同的采样概率，保证不同类别的图片在同一个 minibatch 或者同一个 epoch 内，不同类别的训练样本数量基本一致或者符合自己期望的比例。 + * 可以使用过采样的方法，对图片数量较少的类别进行过采样。 +* 从损失函数的角度出发的话 + * 可以使用 OHEM(online hard example miniing)的方法，对根据样本的 loss 进行筛选，筛选出 hard example 用于模型的梯度反传和参数更新。 + * 可以使用 Focal loss 的方法，对一些比较容易的样本的 loss 赋予较小的权重，对于难样本的 loss 赋予较大的权重，从而让容易样本的 loss 对网络整体的 loss 有贡献，但是又不会主导 loss。 + + +### Q1.3 在 docker 中训练的时候，数据路径和配置均没问题，但是一直报错 `SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception`，这是为什么呢？ + +**A**：这可能是因为 docker 中共享内存太小导致的。创建 docker 的时候，`/dev/shm` 的默认大小为 64M，如果使用多进程读取数据，共享内存可能不够，因此需要给 `/dev/shm` 分配更大的空间，在创建 docker 的时候，传入 `--shm-size=8g` 表示给 `/dev/shm` 分配 8G 的空间，一般是够用的。 + + +### Q1.4 PaddleClas 提供的 10W 类图像分类预训练模型在哪里下载，应该怎么使用呢？ + +**A**：基于 ResNet50_vd, 百度开源了自研的大规模分类预训练模型，其中训练数据为 10 万个类别，4300 万张图片。10 万类预训练模型的下载地址：[下载地址](https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_10w_pretrained.tar)，在这里需要注意的是，该预训练模型没有提供最后的 FC 层参数，因此无法直接拿来预测；但是可以使用它作为预训练模型，在自己的数据集上进行微调。经过验证，该预训练模型相比于基于 ImageNet1k 数据集的 ResNet50_vd 预训练模型，在不同的数据集上均有比较明显的精度收益，最多可达 30%。 + + + + +### Q1.5 使用 C++ 进行预测部署的时候怎么进行加速呢？ + +**A**：可以从以下几个方面加速预测过程。 + +1. 如果是 CPU 预测的话，可以开启 mkldnn 进行预测，同时适当增大运算的线程数(cpu_math_library_num_threads，在 `tools/config.txt` 中)，一般设置为 6~10 比较有效。 +2. 如果是 GPU 预测的话，在硬件条件允许的情况下，可以开启 TensorRT 预测以及 FP16 预测，这可以进一步加快预测速度。 +3. 在内存或者显存足够的情况下，可以增大预测的 batch size。 +4. 可以将图像预处理的逻辑(主要设计 resize、crop、normalize 等)放在 GPU 上运行，这可以进一步加速预测过程。 + +更多的预测部署加速技巧，也欢迎大家补充。 + + +## 第 2 期 + +### Q2.1: PaddleClas 在设置标签的时候必须从 0 开始吗？class_num 必须等于数据集的类别数吗？ + +**A**：在 PaddleClas 中，标签默认是从 0 开始，所以，尽量从 0 开始设置标签，当然，从其他值开始设置也可以，这样会导致设置的 class_num 增大，进而导致分类的 FC 层参数量较大，权重文件会占用更多的存储空间。在数据集类别连续的情况下，设置的 class_num 要等于数据集类别数（当然大于数据集类别数也可以，在很多数据集上甚至可以获得更高的精度，但同样会使 FC 层参数量较大），在数据集类别数不连续的情况下，设置的 class_num 要等于数据集中最大的 class_id+1。 + +### Q2.2: 当类别数特别多的时候，最后的 FC 特别大，导致权重文件占用较大的存储空间，该怎么解决？ + +**A**：最终的 FC 的权重是一个大的矩阵，大小为 C*class_num，其中 C 为 FC 前一层的神经单元个数，如 ResNet50 中的 C 为 2048，可以通过降低 C 的值来进一步减小 FC 权重的大小，比如，可以在 GAP 之后加一层维数较小的 FC 层，这样可以大大缩小最终分类层的权重大小。 + +### Q2.3: 为什么使用 PaddleClas 在自定义的数据集上训练 ssld 蒸馏没有达到预期？ + +首先，需要确保 Teacher 模型的精度是否存在问题，其次，需要确保 Student 模型是否成功加载了 ImageNet-1k 的预训练权重以及 Teacher 模型是否成功加载了训练自定义数据集的权重，最后，要确保初次学习率不应太大，至少保证初始学习率不要超过训练 ImageNet-1k 的值。 + +### Q2.4: 移动端或嵌入式端上哪些网络具有优势？ + +建议使用移动端系列的网络，网络详情可以参考[移动端系列网络结构介绍](../models/Mobile.md)。如果任务的速度更重要，可以考虑 MobileNetV3 系列，如果模型大小更重要，可以根据移动端系列网络结构介绍中的 StorageSize-Accuracy 来确定具体的结构。 + +### Q2.5: 既然移动端网络非常快，为什么还要使用诸如 ResNet 这样参数量和计算量较大的网络？ + +不同的网络结构在不同的设备上运行速度优势不同。在移动端，移动端系列的网络比服务器端的网络运行速度更快，但是在服务器端，相同精度下，ResNet 等经过特定优化后的网络具有更大的优势，所以需要根据具体情况来选择具体的网络结构。 + + +## 第 3 期 + +### Q3.1: 双（多）分支结构与 Plain 结构，各自有什么特点？ + +**A**： +以 VGG 为代表的 Plain 网络，发展到以 ResNet 系列（带有残差模块）、Inception 系列（多卷积核并行）为代表的的多分支网络结构，人们发现多分支结构在模型训练阶段更为友好，更大的网络宽度可以带来更强的特征拟合能力，而残差结构则可以避免深度网络梯度消失的问题，但是在推理阶段，带有多分支结构的模型在速度上并无优势，即使多分支结构模型的 FLOPs 要更低，但多分支结构的模型计算密度也更低。例如 VGG16 模型的 FLOPs 远远大于 EfficientNetB3，但是 VGG16 模型的推理速度却显著快于 EfficientNetB3，因此多分支结构在模型训练阶段更为友好，而 Plain 结构模型则更适合于推理阶段，那么以此为出发点，可以在训练阶段使用多分支网络结构，以更大的训练时间成本换取特征拟合能力更强的模型，而在推理阶段，将多分支结构转为 Plain 结构，从而换取更短的推理时间。实现多分支结构到 Plain 结构的转换，可以通过结构重参数化(structural re-parameterization)技术实现。 + +另外，Plain 结构对于剪枝操作也更为友好。 + +注：“Plain 结构”与“结构重参数化(structural re-parameterization)技术”出自论文“RepVGG: Making VGG-style ConvNets Great Again”。Plain 结构网络模型指整个网络不存在分支结构，也即网络中第 `i` 层 layer 的输入为第 `i-1` 层 layer 的输出，第 `i` 层 layer 的输出为第 `i+1` 层 layer 的输入。 + +### Q3.2: ACNet 的创新点主要在哪里？ +**A**： +ACNet 意为“Asymmetric Convolution Block”，即为非对称卷积模块，该思想出自论文“ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks”，文章提出了以“ACB”结构的三个 CNN 卷积核为一组，用来在训练阶段替代现有卷积神经网络中的传统方形卷积核。 + +方形卷积核的尺寸为假设为 `d*d`，即宽、高相等均为 `d`，则用于替换该卷积核的 ACB 结构是尺寸为 `d*d`、`1*d`、`d*1` 的三个卷积核，然后再将三个卷积核的输出直接相加，可以得到与原有方形卷积核相同尺寸的计算结果。 +而在训练完成后，将 ACB 结构换回原有的方形卷积核，方形卷积核的参数则为 ACB 结构的三个卷积核的参数直接相加（见 `Q3.4`，因此还是使用与之前相同的模型结构用于推理，ACB 结构只是在训练阶段使用。 + +在训练中，通过 ACB 结构，模型的网络宽度得到了提高，利用 `1*d`、`d*1` 的两个非对称卷积核提取得到更多的特征用于丰富 `d*d` 卷积核提取的特征图的信息。而在推理阶段，这种设计思想并没有带来额外的参数与计算开销。如下图所示，分别是用于训练阶段和部署推理阶段的卷积核形式。 + +![](../../images/faq/TrainingtimeACNet.png) + +![](../../images/faq/DeployedACNet.png) + +文章作者的实验表明，通过在原有网络模型训练中使用 ACNet 结构可以显著提高模型能力，原作者对此有如下解释： + +1. 实验表明，对于一个 `d*d` 的卷积核，相对于消除卷积核角落位置（如上图中卷积核的 `corners` 位置）的参数而言，消除骨架位置（如上图中卷积核的 `skeleton` 位置）的参数会给模型精度带来更大的影响，因此卷积核骨架位置的参数要更为重要，而 ACB 结构中的两个非对称卷积核增强了方形卷积核骨架位置参数的权重，使之作用更为显著。这种相加是否会因正负数抵消作用而减弱骨架位置的参数作用，作者通过实验发现，网络的训练总是会向着提高骨架位置参数作用的方向发展，并没有出现正负数抵消而减弱的现象。 +2. 非对称卷积核对于翻转的图像具有更强的鲁棒性，如下图所示，水平的非对称卷积核对于上下翻转的图像具有更强的鲁棒性。对于翻转前后图像中语义上的同一位置，非对称卷积核提取的特征图是相同的，这一点要强于方形卷积核。 + +![](../../images/faq/HorizontalKernel.png) + +### Q3.3: RepVGG 的创新点主要在哪里？ + +**A**： +通过 Q3.1 与 Q3.2，我们可以大胆想到，是否可以借鉴 ACNet 将训练阶段与推理阶段解耦，并且训练阶段使用多分支结构，推理阶段使用 Plain 结构，这也就是 RepVGG 的创新点。下图为 ResNet、RepVGG 训练和推理阶段网络结构的对比。 + +![](../../images/faq/RepVGG.png) + +首先训练阶段的 RepVGG 采用多分支结构，可以看作是在传统 VGG 网络的基础上，增加了 `1*1` 卷积和恒等映射的残差结构，而推理阶段的 RepVGG 则退化为 VGG 结构。训练阶段 RepVGG 到推理阶段 RepVGG 的网络结构转换使用“结构重参数化”技术实现。 + +对于恒等映射，可将其视为参数均为 `1` 的 `1*1` 卷积核作用在输入特征图的输出结果，因此训练阶段的 RepVGG 的卷积模块可以视为两个 `1*1` 卷积和一个 `3*3` 卷积，而 `1*1` 卷积的参数又可以直接相加到 `3*3` 卷积核中心位置的参数上（该操作类似于 ACNet 中，非对称卷积核参数相加到方形卷积核骨架位置参数的操作），通过上述操作，即可在推理阶段，将网络结构中的恒等映射、`1*1 `卷积、`3*3` 卷积三个分支合并为一个 `3*3` 卷积，详见 `Q3.4`。 + +### Q3.4: ACNet 与 RepVGG 中的 struct re-parameters 有何异同？ + +**A**： +通过上面的了解，可以简单理解 RepVGG 是更为极端的 ACNet。ACNet 中的 re-parameters 操作如下图所示： + +![](../../images/faq/ACNetReParams.png) + +观察上图，以其中的 `conv2` 为例，该非对称卷积可以视为 `3*3` 的方形卷积核，只不过该方形卷积核的上下六个参数为 `0`，`conv3` 同理。并且，`conv1`、`conv2`、`conv3` 的结果相加，等同于三个卷积核相加再做卷积，以 `Conv` 表示卷积操作，`+`表示矩阵的加法操作，则：`Conv1(A)+Conv2(A)+Conv3(A) == Convk(A)`，其中 `Conv1`、`Conv2`、`Conv3` 的卷积核分别为 `Kernel1`、`kernel2`、`kernel3`，而 `Convk` 的卷积核为 `Kernel1 + kernel2 + kernel3`。 + +RepVGG 网络与 ACNet 同理，只不过 ACNet 的 `1*d` 非对称卷积变成了 `1*1` 卷积，`1*1` 卷积相加的位置变成了 `3*3` 卷积的中心。 + +### Q3.5: 影响模型计算速度的因素都有哪些？参数量越大的模型计算速度一定更慢吗？ + +**A**： +影响模型计算速度的因素有很多，参数量只是其中之一。具体来说，在不考虑硬件差异的前提下，模型的计算速度可以参考以下几个方面： +1. 参数量：用于衡量模型的参数数量，模型的参数量越大，模型在计算时对内存（显存）的容量要求一般也更高。但内存（显存）占用大小不完全取决于参数量。如下图中，假设输入特征图内存占用大小为 `1` 个单位，对于左侧的残差结构而言，由于需要记录两个分支的运算结果，然后再相加，因此该结构在计算时的内存峰值占用是右侧 Plain 结构的两倍。 + +![](../../images/faq/MemoryOccupation.png) + +2. 浮点运算数量（FLOPs）：注意与每秒浮点运算次数（FLOPS）相区分。FLOPs 可以简单理解为计算量，通常用来衡量一个模型的计算复杂度。 +以常见的卷积操作为例，在不考虑 batch size、激活函数、stride 操作、bias 的前提下，假设 input future map 尺寸为 `Min*Min`，通道数为 `Cin`，output future map 尺寸为 `Mout*Mout`，通道数为 `Cout`，conv kernel 尺寸为 `K*K`，则进行一次卷积的 FLOPs 可以通过下述方式计算： + 1. 输出特征图包含特征点的数量为：`Cout * Mout * Mout`； + 2. 对于输出特征图中的每一个特征点的卷积操作而言： + 乘法计算数量为：`Cin * K * K`； + 加法计算数量为：`Cin * K * K - 1`； + 3. 因此计算总量为：`Cout * Mout * Mout * (Cin * K * K + Cin * K * K - 1)`，也即 `Cout * Mout * Mout * (2Cin * K * K - 1)`。 +3. Memory Access Cost（MAC）：内存访问成本，由于计算机在对数据进行运算（例如乘法、加法）前，需要将运算的数据从内存（此处泛指内存，包括显存）读取到运算器的 Cache 中，而内存的访问是十分耗时的。以分组卷积为例，假设分为 `g` 组，虽然分组后模型的参数量和 FLOPs 没有变化，但是分组卷积的内存访问次数成为之前的 `g` 倍（此处只是简单计算，未考虑多级 Cache），因此 MAC 显著提高，模型的计算速度也相应变慢。 +4. 并行度：常说的并行度包括数据并行和模型并行两部分，此处是指模型并行。以卷积操作为例，一个卷积层的参数量通常十分庞大，如果将卷积层中的矩阵做分块处理，然后分别交由多个 GPU 进行运算，即可达到加速的目的。甚至有的网络层参数量过大，单张 GPU 显存无法容纳时，也可能将该层分由多个 GPU 计算，但是能否分由多个 GPU 并行运算，不仅取决于硬件条件，也受特定的运算形式所限制。当然，并行度越高的模型，其运行速度也越快。 + + + +## 第 4 期 + +### Q4.1: 图像分类任务中，有一部分合成数据，这一部分合成数据是否需要使用样本均衡？ + +**A**: + +1. 不同类别的样本数如果差异过大，某类样本因合成数据集扩充至其他类的数倍以上，需要做适当减小该类的权值。 +2. 如果是有的类别是合成而有的类别半合成半真实，只要数量在一个量级可不做均衡，并尝试训练一下，测试该合成类别样本是否能够准确识别。 +3. 如果不同来源数据的类别因合成数据增大问题，造成性能衰减，需要考虑合成数据集是否有噪声或者难例样本，也可适当增加该类别权重，获得该类别更好的识别性能。 + +### Q4.2: 学术界将 Vision Transformer(ViT)引入图像分类领域，将对图像分类带来什么新的机遇和挑战？相比于 CNN 的优势？ + +论文地址[AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE](https://openreview.net/pdf?id=YicbFdNTTy) + +**A**: +1. 图像对 CNN 的依赖是不必要的，利用 Transformer 的计算效率和可伸缩性，可以训练很大模型，当模型和数据集增大的情形下，仍然不会存在饱和的情况。受到 Transformer 在 NLP 上的启发，在图像分类任务中使用时，将图片分成顺序排列的 patches，并将这些 patches 输入一个线性单元嵌入到 embedding 作为 transformer 的输入。 + +2. 在中等规模数据集中如 ImageNet1k，ImageNet21k，视觉 Transformer 模型低于相同规模尺寸的 ResNet 几个百分点。猜测这是因为 transformer 缺少 CNN 所具有的局部性(Locality)和空间不变性(Spatial Invariance)的特点，而在数据量不够大的时候，难以超越卷积网络，不过对于这一问题，[DeiT](https://arxiv.org/abs/2012.12877) 使用数据增强的方式在一定程度上解决了 Vision Transformer 依赖超大规模数据集训练的问题。 + +3. 在超大规模数据集 14M-300M 训练时，这种方式可以越过局部信息，建模更加长距离的依赖关系，而 CNN 能较好关注局部信息全局信息捕获能力较弱。 + +4. Transformer 一度在 NLP 领域一统江湖，也一度被质疑不适用于 CV 领域，当前出来的几片视觉领域的文章，性能也是直逼 CNN 的 SOTA。相信在未来能够提出 Vision-Language 联合或者多模态的模型，能够一并解决视觉和语言问题。 + + +### Q4.3: 对于 Vision Transformer 模型，是如何将图像转换成序列信息传给 Encoder？ + +**A**: + +1. 使用 Transformer 模型，主要是使用其中的注意力方法。我们希望构想一个适用语义 embedding 信息的场景，但是图像分类与序列的语义信息关联性不大，因此 Vision Transformer 有其独特的设计。ViT 的目标也正是希望使用注意力机制来代替 CNN。 + +2. 考虑 Transformer 中 Encoder 编码器的输入形式，如下图: + * (1)不定长度的顺序输入，因为它是 RNN 结构，一句话，单词数不一样。如果是 NLP 场景，换词的顺序不太过于影响语义，但是图像换了不同区域的位置，不同区域连接顺序不同，将造成极大理解偏差。 + * (2)单个 patch 位置信息通过变换成一个维度固定的向量，Encoder 输入是 patch 像素信息 embedding，与一些固定位置的向量 concate，合成一个维度固定的向量和位置信息在其中。 + +![](../../images/faq/Transformer_input.png) + +3. 考虑以下问题：怎样将一张图片传给 encoder？ + +* 如下图所示。假设输入图片是[224,224,3]，按照顺序从左到右，从上到下，切分成很多个 patch，patch 大小可以为[p,p,3]（p 取值可以是 16，32），对其使用 Linear Projection of Flattened Patches 模块转成特征向量，并 concat 一个位置向量，传入 Encoder 中。 + +![](../../images/faq/ViT_structure.png) + +4. 如上图，给定一个 `H×W×C` 的图像以及区块大小 P，可以把图像划分为 `N` 个 `P×P×C` 的区块，`N=H×W/(P×P)`。得到区块后要使用线性变换转为 D 维特征向量，再加上位置编码向量即可。和 BERT 类似，ViT 在序列之前也加入了一个分类标志位，记为 `[CLS]`。ViT 输入序列 `z` 如下面的公式所示，其中 `x` 表示一个图像区块。 + +![](../../images/faq/ViT.png) + +5. ViT 模型和 Transformer 基本一样，输入序列传入 ViT，然后利用 `[CLS]` 标志位的最终输出特征进行分类。ViT 主要由 MSA(多头自注意力)和 MLP(两层使用 GELU 激活函数的全连接网络)组成，在 MSA 和 MLP 之前加上 LayerNorm 和残差连接。 + +### Q4.4: 如何理解归纳偏置 Inductive Bias？ + +**A**: +1. 在机器学习中，会对算需要应用的问题做一些假设，这个假设就称为归纳偏好。在现实生活中观察得到的现象中归纳出一定的先验规则，然后对模型做一定的约束，从而起到模型选择的作用。在 CNN 中，假设特征具有局部性(Locality)和空间不变性(Spatial Invariance)的特点，即把相邻的特征有联系而远离的没有，将相邻特征融合在一起，更会容易产生“解”；还有 attention 机制，也是从人的直觉、生活经验归纳的规则。 + +2. Vision Transformer 利用的归纳偏置是有序列能力 Sequentiality 和时间不变性 Time Invariance，即序列顺序上的时间间隔的联系，因此也能得出在更大规模数据集上比 CNN 类的模型有更好的性能。文章 Conclusion 里的“Unlike prior works using self-attention in computer vision, we do not introduce any image-specific inductive biases into the architecture”和 Introduction 里的“We find that large scale training trumps inductive bias”，可以得出直观上 inductive bias 在大量数据的情况中的产生是衰减性能，应该尽可能丢弃。 + +### Q4.5: ViT 为什么要增加一个[CLS]标志位? 为什么将[CLS]标志位对应的向量作为整个序列的语义表示? + +**A**: +1. 和 BERT 相类似，ViT 在第一个 patch 前添加一个`[CLS]`标志位，最后一个结束标志位对应的向量可以作为整个图像的语义表示，从而用于下游的分类任务等。从而使得整个 embedding 组可以表征该图像不同位置的特征。 + +2. 将 `[CLS]` 标志位对应的向量作为整个图像的语义表示，是因为与图像中已有的其它 patch 块图像相比，这个无明显语义信息的符号会更“公平”地融合图像中各个 patch 的语义信息，从而更好的表示整个图像的语义。 + + +## 第 5 期 + +### Q5.1: PaddleClas 训练配置文件中都包含了哪些内容？训练模型时如何进行修改？ +**A**: +PaddleClas 的模型包含 6 大模块的配置，分别为：全局配置，网络结构(ARCHITECTURE)，学习率(LEARNING_RATE)，优化器(OPTIMIZER)，训练(TRAIN)和验证(VALID)。 + +全局配置主要包含了任务的配置信息，例如类别的数量，训练集内的数据量，训练的 epoch 数量，网络输入的尺寸等等。如果要训练自定义的任务，或是使用自己的训练集，需要关注这个部分。 + +网络结构的配置定义了要使用的网络。在实际使用时，首先要选择合适的配置文件，所以通常不会修改这部分配置。只有在自定义网络结构，或者对任务有特殊要求时才进行修改。 + +学习率和优化器的配置建议优先使用默认配置，这些参数是我们已经调过的。如果任务的改动比较大，也可以做微调。 + +训练和预测两个配置包含了 batch_size，数据集，数据预处理(transforms)，读数据进程数(num_workers)等比较重要的配置，这部分要根据实际环境适度修改。要注意的是，paddleclas 中的 batch_size 是单卡配置，如果是多卡训练，则总的 batch_size 是配置文件中所设置的倍数，例如配置文件中设置 batch_size 为 64，4 卡训练，总 batch_size 也就是 4*64=256。而 num_workers 定义的是单卡的进程数，即如果 num_workers 是 8，并且使用 4 卡训练，则实际有 32 个 worker. + +### Q5.2: 如何在命令行中快速的修改配置？ +**A**: +在训练中，我们常常需要对个别配置不断进行微调，而不希望频繁的修改配置文件。这时可以使用 -o 来调整，修改是要先按层级写出要改的配置名称，层级之间用点分割，再写出要修改的值。例如我们想要修改 batch_size，可以在训练的命令后加上-o DataLoader.TRAIN.sampler.batch_size=512。 + +### Q5.3: 如何根据 PaddleClas 的精度曲线选择合适的模型？ +**A**: +PaddleClas 提供了多个模型的 benchmark，并绘制了性能曲线，主要有三种：准确率-预测时间曲线，准确率-参数量曲线和准确率-FLOPS 曲线，纵轴为准确率，横轴分别为预测时间、参数量、FLOPS。一般来说，不同模型在三个图上的表现是一致的。同一个系列的模型在图上采用相同的符号来表示，并且用曲线连接。 + +以准确率-预测时间曲线为例，点越靠上表明准确率越高，约靠左表明速度越快。例如左上区域的模型为又快又准的模型，而最左侧贴近纵轴的点则为轻量级的模型。使用时，可以综合考虑准确率和时间因素，选择合适的模型。举个例子，我们希望模型的运行时间在 10ms 以下，选择最准确的模型。首先，在横轴的 10ms 出画一条垂直的线，之后在这条线的左侧找到最高的点，就是符合要求的模型。 + +实际使用中，模型的参数量和 FLOPS 是稳定不变的，而运算时间在不同的软硬件条件下均会有所变化。如果想更准确的选择模型，那么可以在自己的环境中运行测试，得到该环境下的性能图。 + +### Q5.4: 想在 imagenet 中增加两个类，能不能把现有全连接层的参数固定，只训练新的两个类别？ +**A**: +这个想法理论上可行，但效果恐怕不会太好。如果只是固定全连接层，而前面的卷积层参数发生了变化，那么这些全连接层的作用也无法保证与开始一样。而如果保持整个网络的参数都不变，只训练全连接层的新增两个类别，也比较难训练处理想的结果。 + +如果实际使用中确实需要原有的 1000 个类别依然很准确，那么可以将新类别的数据加入到原有训练集中，再用预训练模型进行 finetune。如果只需要 1000 个类中的几个类别，那么可以把这部分的数据挑出来，和新增数据混合再 finetune。 + +### Q5.5: 使用分类模型做其他任务的预训练模型时，应该选择哪些层作为 feature？ +**A**: +使用分类模型做其他任务的 backbone 有很多策略，这里介绍一种较为基础的方法。首先，去掉最后的全连接层，这一层主要包含的是原始任务的分类信息。如果任务比较简单，只要将前一层的输出作为 featuremap，并在此基础上添加与任务对应的结构即可。如果任务涉及多尺度，需要选取不同尺度的 anchor，例如某些检测模型，那么可以选取每次下采样之前一层的输出作为 featuremap。 diff --git a/src/PaddleClas/docs/zh_CN/faq_series/faq_2021_s2.md b/src/PaddleClas/docs/zh_CN/faq_series/faq_2021_s2.md new file mode 100644 index 0000000..38b9d2d --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/faq_series/faq_2021_s2.md @@ -0,0 +1,264 @@ +# PaddleClas 相关常见问题汇总 - 2021 第 2 季 + +## 写在前面 + +* 我们收集整理了开源以来在 issues 和用户群中的常见问题并且给出了简要解答，旨在为广大用户提供一些参考，也希望帮助大家少走一些弯路。 + +* 图像分类、识别、检索领域大佬众多，模型和论文更新速度也很快，本文档回答主要依赖有限的项目实践，难免挂一漏万，如有遗漏和不足，也希望有识之士帮忙补充和修正，万分感谢。 + +## 目录 + +* [1. 理论篇](#1) + * [1.1 PaddleClas 基础知识](#1.1) + * [1.2 骨干网络和预训练模型库](#1.2) + * [1.3 图像分类](#1.3) + * [1.4 通用检测模块](#1.4) + * [1.5 图像识别模块](#1.5) + * [1.6 检索模块](#1.6) +* [2. 实战篇](#2) + * [2.1 训练与评估共性问题](#2.1) + * [2.2 图像分类](#2.2) + * [2.3 通用检测模块](#2.3) + * [2.4 图像识别模块](#2.4) + * [2.5 检索模块](#2.5) + * [2.6 模型预测部署](#2.6) + + +## 1. 理论篇 + + +### 1.1 PaddleClas 基础知识 + +#### Q1.1.1 PaddleClas 和 PaddleDetection 区别 +**A**：PaddleClas 是一个兼主体检测、图像分类、图像检索于一体的图像识别 repo，用于解决大部分图像识别问题，用户可以很方便的使用 PaddleClas 来解决小样本、多类别的图像识别问题。PaddleDetection 提供了目标检测、关键点检测、多目标跟踪等能力，方便用户定位图像中的感兴趣的点和区域，被广泛应用于工业质检、遥感图像检测、无人巡检等项目。 + +#### Q1.1.3: Momentum 优化器中的 momentum 参数是什么意思呢？ +**A**: Momentum 优化器是在 SGD 优化器的基础上引入了“动量”的概念。在 SGD 优化器中，在 `t+1` 时刻，参数 `w` 的更新可表示为： +```latex +w_t+1 = w_t - lr * grad +``` +其中，`lr` 为学习率，`grad` 为此时参数 `w` 的梯度。在引入动量的概念后，参数 `w` 的更新可表示为： +```latex +v_t+1 = m * v_t + lr * grad +w_t+1 = w_t - v_t+1 +``` +其中，`m` 即为动量 `momentum`，表示累积动量的加权值，一般取 `0.9`，当取值小于 `1` 时，则越早期的梯度对当前的影响越小，例如，当动量参数 `m` 取 `0.9` 时，在 `t` 时刻，`t-5` 的梯度加权值为 `0.9 ^ 5 = 0.59049`，而 `t-2` 时刻的梯度加权值为 `0.9 ^ 2 = 0.81`。因此，太过“久远”的梯度信息对当前的参考意义很小，而“最近”的历史梯度信息对当前影响更大，这也是符合直觉的。 + +![](../../images/faq/momentum.jpeg) + +通过引入动量的概念，在参数更新时考虑了历史更新的影响，因此可以加快收敛速度，也改善了 `SGD` 优化器带来的损失（cost、loss）震荡问题。 + +#### Q1.1.4: PaddleClas 是否有 `Fixing the train-test resolution discrepancy` 这篇论文的实现呢？ +**A**: 目前 PaddleClas 没有实现。如果需要，可以尝试自己修改代码。简单来说，该论文所提出的思想是使用较大分辨率作为输入，对已经训练好的模型最后的 FC 层进行 fine-tune。具体操作上，首先在较低分辨率的数据集上对模型网络进行训练，完成训练后，对网络除最后的 FC 层外的其他层的权重设置参数 `stop_gradient=True`，然后使用较大分辨率的输入对网络进行 fine-tune 训练。 + + +### 1.2 骨干网络和预训练模型库 + + +### 1.3 图像分类 + +#### Q1.3.1: PaddleClas 有提供调整图片亮度，对比度，饱和度，色调等方面的数据增强吗？ +**A**：PaddleClas 提供了多种数据增广方式，可分为 3 类： +1. 图像变换类： AutoAugment, RandAugment; +2. 图像裁剪类： CutOut、RandErasing、HideAndSeek、GridMask； +3. 图像混叠类：Mixup, Cutmix. + +其中，RandAngment 提供了多种数据增强方式的随机组合，可以满足亮度、对比度、饱和度、色调等多方面的数据增广需求。 + + +### 1.4 通用检测模块 + +#### Q1.4.1 主体检测是每次只输出一个主体检测框吗？ +**A**：主体检测这块的输出数量是可以通过配置文件配置的。在配置文件中 Global.threshold 控制检测的阈值，小于该阈值的检测框被舍弃，Global.max_det_results 控制最大返回的结果数，这两个参数共同决定了输出检测框的数量。 + +#### Q1.4.2 训练主体检测模型的数据是如何选择的？换成更小的模型会有损精度吗？ +**A**：训练数据是在 COCO、Object365、RPC、LogoDet 等公开数据集中随机抽取的子集。目前我们在 2.3 版本中推出了超轻量的主体检测模型，具体信息可以参考[主体检测](../image_recognition_pipeline/mainbody_detection.md#2-模型选择)。关于主体检测模型的更多信息请参考[主体检测](../image_recognition_pipeline/mainbody_detection.md)。 + +#### Q1.4.3: 目前使用的主体检测模型检测在某些场景中会有误检？ +**A**：目前的主体检测模型训练时使用了 COCO、Object365、RPC、LogoDet 等公开数据集，如果被检测数据是类似工业质检等于常见类别差异较大的数据，需要基于目前的检测模型重新微调训练。 + + +### 1.5 图像识别模块 + +#### Q1.5.1 使用 `circle loss` 还需加 `triplet loss` 吗？ +**A**：`circle loss` 是统一了样本对学习和分类学习的两种形式，如果是分类学习的形式的话，可以增加 `triplet loss`。 + +#### Q1.5.2 如果不是识别开源的四个方向的图片，该使用哪个识别模型？ +**A**：建议使用商品识别模型，一来是因为商品覆盖的范围比较广，被识别的图片是商品的概率更大，二来是因为商品识别模型的训练数据使用了 5 万类别的数据，泛化能力更好，特征会更鲁棒一些。 + +#### Q1.5.3 最后使用 512 维的向量，为什么不用 1024 或者其他维度的呢？ +**A**：使用维度小的向量，为了加快计算，在实际使用过程中，可能使用 128 甚至更小。一般来说，512 的维度已经够大，能充分表示特征了。 + + +### 1.6 检索模块 + +#### Q1.6.1 PaddleClas 目前使用的 Möbius 向量检索算法支持类似于 faiss 的那种 index.add()的功能吗? 另外，每次构建新的图都要进行 train 吗？这里的 train 是为了检索加速还是为了构建相似的图？ +**A**：目前在 release/2.3 分支已经支持 faiss 检索模块，并且不再支持 Möbius。关于 Möbius 提供的检索算法，是一种基于图的近似最近邻搜索算法，目前支持两种距离计算方式：inner product 和 L2 distance，但是 Möbius 暂不支持 faiss 中提供的 index.add 功能，如果需要增加检索库的内容，需要从头重新构建新的 index. 在每次构建 index 时，检索算法内部执行的操作是一种类似于 train 的过程，不同于 faiss 提供的 train 接口。因此需要 faiss 模块的话，可以使用 release/2.3 分支，需要 Möbius 的话，目前需要回退到 release/2.2 分支。 + +#### Q1.6.2: PaddleClas 图像识别用于 Eval 的配置文件中，`Query` 和 `Gallery` 配置具体是用于做什么呢？ +**A**: `Query` 与 `Gallery` 均为数据集配置，其中 `Gallery` 用于配置底库数据，`Query` 用于配置验证集。在进行 Eval 时，首先使用模型对 `Gallery` 底库数据进行前向计算特征向量，特征向量用于构建底库，然后模型对 `Query` 验证集中的数据进行前向计算特征向量，再与底库计算召回率等指标。 + + +## 2. 实战篇 + + +### 2.1 训练与评估共性问题 + +#### Q2.1.1 PaddleClas 的 `train_log` 文件在哪里? +**A**：在保存权重的路径中存放了 `train.log`。 + +#### Q2.1.2 模型训练出 nan，为什么？ +**A**： +1.确保正确加载预训练模型, 最简单的加载方式添加参数 `-o Arch.pretrained=True` 即可； +2.模型微调时，学习率不要太大，如设置 0.001 就好。 + +#### Q2.1.3 可以对视频中每一帧画面进行逐帧预测吗？ +**A**：可以，但目前 PaddleClas 并不支持视频输入。可以尝试修改一下 PaddleClas 代码，或者预先将视频逐帧转为图像存储，再使用 PaddleClas 进行预测。 + +#### Q2.1.4: 数据预处理中，不想对输入数据进行裁剪，该如何设置？或者如何设置剪裁的尺寸。 +**A**: PaddleClas 支持的数据预处理算子可在这里查看：`ppcls/data/preprocess/__init__.py`，所有支持的算子均可在配置文件中进行配置，配置的算子名称需要和算子类名一致，参数与对应算子类的构造函数参数一致。如不需要对图像裁剪，则可去掉 `CropImage`、`RandCropImage`，使用 `ResizeImage` 替换即可，可通过其参数设置不同的 resize 方式，使用 `size` 参数则直接将图像缩放至固定大小，使用 `resize_short` 参数则会维持图像宽高比进行缩放。设置裁剪尺寸时，可通过 `CropImage` 算子的 `size` 参数，或 `RandCropImage` 算子的 `size` 参数。 + +#### Q2.1.5: PaddlePaddle 安装后，使用报错，无法导入 paddle 下的任何模块（import paddle.xxx），是为什么呢？ +**A**: 首先可以使用以下代码测试 Paddle 是否安装正确： +```python +import paddle +paddle.utils.install_check.run_check(） +``` +正确安装时，通常会有如下提示： +``` +PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now. +``` +如未能安装成功，则会有相应问题的提示。 +另外，在同时安装 CPU 版本和 GPU 版本 Paddle 后，由于两个版本存在冲突，需要将两个版本全部卸载，然后重新安装所需要的版本。 + +#### Q2.1.6: 使用 PaddleClas 训练时，如何设置仅保存最优模型？不想保存中间模型。 +**A**: PaddleClas 在训练过程中，会保存/更新以下三类模型： +1. 最新的模型（`latest.pdopt`，`latest.pdparams`，`latest.pdstates`），当训练意外中断时，可使用最新保存的模型恢复训练； +2. 最优的模型（`best_model.pdopt`，`best_model.pdparams`，`best_model.pdstates`）； +3. 训练过程中，一个 epoch 结束时的断点（`epoch_xxx.pdopt`，`epoch_xxx.pdparams`，`epoch_xxx.pdstates`）。训练配置文件中 `Global.save_interval` 字段表示该模型的保存间隔。将该字段设置大于总 epochs 数，则不再保存中间断点模型。 + +#### Q2.1.7: 在训练时，出现如下报错信息：`ERROR: Unexpected segmentation fault encountered in DataLoader workers.`，如何排查解决问题呢？ +**A**：尝试将训练配置文件中的字段 `num_workers` 设置为 `0`；尝试将训练配置文件中的字段 `batch_size` 调小一些；检查数据集格式和配置文件中的数据集路径是否正确。 + +#### Q2.1.8: 如何在训练时使用 `Mixup` 和 `Cutmix` ？ +**A**： +* `Mixup` 的使用方法请参考 [Mixup](../../../ppcls/configs/ImageNet/DataAugment/ResNet50_Mixup.yaml#L63-L65)；`Cuxmix` 请参考 [Cuxmix](../../../ppcls/configs/ImageNet/DataAugment/ResNet50_Cutmix.yaml#L63-L65)。 + +* 使用 `Mixup` 或 `Cutmix` 做训练时无法计算训练的精度（Acc）指标，因此需要在配置文件中取消 `Metric.Train.TopkAcc` 字段，可参考 [Metric.Train.TopkAcc](../../../ppcls/configs/ImageNet/DataAugment/ResNet50_Cutmix.yaml#L125-L128)。 + +#### Q2.1.9: 训练配置 yaml 文件中，字段 `Global.pretrain_model` 和 `Global.checkpoints` 分别用于配置什么呢？ +**A**： +* 当需要 `fine-tune` 时，可以通过字段 `Global.pretrain_model` 配置预训练模型权重文件的路径，预训练模型权重文件后缀名通常为 `.pdparams`； +* 在训练过程中，训练程序会自动保存每个 epoch 结束时的断点信息，包括优化器信息 `.pdopt` 和模型权重信息 `.pdparams`。在训练过程意外中断等情况下，需要恢复训练时，可以通过字段 `Global.checkpoints` 配置训练过程中保存的断点信息文件，例如通过配置 `checkpoints: ./output/ResNet18/epoch_18` 即可恢复 18 epoch 训练结束时的断点信息，PaddleClas 将自动加载 `epoch_18.pdopt` 和 `epoch_18.pdparams`，从 19 epoch 继续训练。 + + +### 2.2 图像分类 + +#### Q2.2.1 在 SSLD 中，大模型在 500M 数据上预训练后蒸馏小模型，然后在 1M 数据上蒸馏 finetune 小模型，具体步骤是怎样做的？ +**A**：步骤如下： +1. 基于 facebook 开源的 `ResNeXt101-32x16d-wsl` 模型去蒸馏得到了 `ResNet50-vd` 模型； +2. 用这个 `ResNet50-vd`，在 500W 数据集上去蒸馏 `MobilNetV3`； +3. 考虑到 500W 的数据集的分布和 100W 的数据分布不完全一致，所以这块，在 100W 上的数据上又 finetune 了一下，精度有微弱的提升。 + +#### Q2.2.2 训练 SwinTransformer，loss 出现 nan +**A**：训练 SwinTransformer 时，请使用版本大于等于 `2.1.1` 的 `Paddle`，并且加载我们提供的预训练模型，学习率也不宜过大。 + + +### 2.3 通用检测模块 + +#### Q2.3.1 为什么有一些图片检测出的结果就是原图？ +**A**：主体检测模型会返回检测框，但事实上为了让后续的识别模型更加准确，在返回检测框的同时也返回了原图。后续会根据原图或者检测框与库中的图片的相似度排序，相似度最高的库中图片的标签即为被识别图片的标签。 + +#### Q2.3.2：在直播场景中，需要提供一个直播即时识别画面，能够在延迟几秒内找到特征目标物并用框圈起，这个可以实现吗？ +**A**：要达到实时的检测效果，需要检测速度达到实时性的要求；PP-YOLO 是 Paddle 团队提供的轻量级目标检测模型，检测速度和精度达到了很好的平衡，可以试试 PP-YOLO 来做检测. 关于 PP-YOLO 的使用，可以参照：[PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/configs/ppyolo/README_cn.md)。 + +#### Q2.3.3: 对于未知的标签，加入 gallery dataset 可以用于后续的分类识别（无需训练），但是如果前面的检测模型对于未知的标签无法定位检测出来，是否还是要训练前面的检测模型？ +**A**：如果检测模型在自己的数据集上表现不佳，需要在自己的检测数据集上再 finetune 下 + + +### 2.4 图像识别模块 + +#### Q2.4.1: 识别模块预测时报 `Illegal instruction` 错？ +**A**：如果使用的是 release/2.2 分支，建议更新为 release/2.3 分支，在 release/2.3 分支中，我们使用 faiss 检索模块替换了 Möbius 检索模型，具体可以参考[向量检索教程](../../../deploy/vector_search/README.md)。如仍存在问题，可以在用户微信群中联系我们，也可以在 GitHub 提 issue。 + +#### Q2.4.2: 识别模型怎么在预训练模型的基础上进行微调训练？ +**A**：识别模型的微调训练和分类模型的微调训练类似，识别模型可以加载商品的预训练模型，训练过程可以参考[识别模型训练](../../zh_CN/models_training/recognition.md)，后续我们也会持续细化这块的文档。 + +#### Q2.4.3: 训练 metric learning 时，每个 epoch 中，无法跑完所有 mini-batch，为什么？ +**A**：在训练 metric learning 时，使用的 Sampler 是 DistributedRandomIdentitySampler，该 Sampler 不会采样全部的图片，导致会让每一个 epoch 采样的数据不是所有的数据，所以无法跑完显示的 mini-batch 是正常现象。该问题在 release/2.3 分支已经优化，请更新到 release/2.3 使用。 + +#### Q2.4.4: 有些图片没有识别出结果，为什么？ +**A**：在配置文件（如 inference_product.yaml）中，`IndexProcess.score_thres` 中会控制被识别的图片与库中的图片的余弦相似度的最小值。当余弦相似度小于该值时，不会打印结果。您可以根据自己的实际数据调整该值。 + + +### 2.5 检索模块 + +#### Q2.5.1: 添加图片后建索引报 `assert text_num >= 2` 错？ +**A**：请确保 data_file.txt 中图片路径和图片名称中间的间隔为单个 table，而不是空格。 + +#### Q2.5.2: 新增底库数据需要重新构建索引吗？ +**A**：从 release/2.3 分支起，我们使用 faiss 检索模块替换了 Möbius 检索模型，已经支持在不构建底库的前提下新增底库数据，具体可以参考[向量检索教程](../../../deploy/vector_search/README.md)。 + +#### Q2.5.3: Mac 重新编译 index.so 时报错如下：clang: error: unsupported option '-fopenmp', 该如何处理？ +**A**：如果使用的是 release/2.2 分支，建议更新为 release/2.3 分支，在 release/2.3 分支中，我们使用 faiss 检索模块替换了 Möbius 检索模型，具体可以参考[向量检索教程](../../../deploy/vector_search/README.md)。如仍存在问题，可以在用户微信群中联系我们，也可以在 GitHub 提 issue。 + +#### Q2.5.4: 在 build 检索底库时，参数 `pq_size` 应该如何设置？ +**A**：`pq_size` 是 PQ 检索算法的参数。PQ 检索算法可以简单理解为“分层”检索算法，`pq_size` 是每层的“容量”，因此该参数的设置会影响检索性能，不过，在底库总数据量不太大（小于 10000 张）的情况下，这个参数对性能的影响很小，因此对于大多数使用场景而言，在构建底库时无需修改该参数。关于 PQ 检索算法的更多内容，可以查看相关[论文](https://lear.inrialpes.fr/pubs/2011/JDS11/jegou_searching_with_quantization.pdf)。 + + +### 2.6 模型预测部署 + +#### Q2.6.1: hub serving 方式启动某个模块，怎么添加该模块的参数呢？ +**A**：具体可以参考 [hub serving 参数](../../../deploy/hubserving/clas/params.py)。 + +#### Q2.6.2: 导出inference模型进行预测部署，准确率异常，为什么呢？ +**A**: 该问题通常是由于在导出时未能正确加载模型参数导致的，首先检查模型导出时的日志，是否存在类似下述内容： +``` +UserWarning: Skip loading for ***. *** is not found in the provided dict. +``` +如果存在，则说明模型权重未能加载成功，请进一步检查配置文件中的 `Global.pretrained_model` 字段，是否正确配置了模型权重文件的路径。模型权重文件后缀名通常为 `pdparams`，注意在配置该路径时无需填写文件后缀名。 + +#### Q2.6.3: 如何将模型转为 `ONNX` 格式？ +**A**：Paddle 支持两种转 ONNX 格式模型的方式，且依赖于 `paddle2onnx` 工具，首先需要安装 `paddle2onnx`： + +```shell +pip install paddle2onnx +``` + +* 从 inference model 转为 ONNX 格式模型： + + 以动态图导出的 `combined` 格式 inference model（包含 `.pdmodel` 和 `.pdiparams` 两个文件）为例，使用以下命令进行模型格式转换： + ```shell + paddle2onnx --model_dir ${model_path} --model_filename ${model_path}/inference.pdmodel --params_filename ${model_path}/inference.pdiparams --save_file ${save_path}/model.onnx --enable_onnx_checker True + ``` + 上述命令中： + * `model_dir`：该参数下需要包含 `.pdmodel` 和 `.pdiparams` 两个文件； + * `model_filename`：该参数用于指定参数 `model_dir` 下的 `.pdmodel` 文件路径； + * `params_filename`：该参数用于指定参数 `model_dir` 下的 `.pdiparams` 文件路径； + * `save_file`：该参数用于指定转换后的模型保存目录路径。 + + 关于静态图导出的非 `combined` 格式的 inference model（通常包含文件 `__model__` 和多个参数文件）转换模型格式，以及更多参数说明请参考 paddle2onnx 官方文档 [paddle2onnx](https://github.com/PaddlePaddle/Paddle2ONNX/blob/develop/README_zh.md#%E5%8F%82%E6%95%B0%E9%80%89%E9%A1%B9)。 + +* 直接从模型组网代码导出 ONNX 格式模型： + + 以动态图模型组网代码为例，模型类为继承于 `paddle.nn.Layer` 的子类，代码如下所示： + + ```python + import paddle + from paddle.static import InputSpec + + class SimpleNet(paddle.nn.Layer): + def __init__(self): + pass + def forward(self, x): + pass + + net = SimpleNet() + x_spec = InputSpec(shape=[None, 3, 224, 224], dtype='float32', name='x') + paddle.onnx.export(layer=net, path="./SimpleNet", input_spec=[x_spec]) + ``` + 其中： + * `InputSpec()` 函数用于描述模型输入的签名信息，包括输入数据的 `shape`、`type` 和 `name`（可省略）； + * `paddle.onnx.export()` 函数需要指定模型组网对象 `net`，导出模型的保存路径 `save_path`，模型的输入数据描述 `input_spec`。 + + 需要注意，`paddlepaddle` 版本需大于 `2.0.0`。关于 `paddle.onnx.export()` 函数的更多参数说明请参考 [paddle.onnx.export](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/onnx/export_cn.html#export)。 diff --git a/src/PaddleClas/docs/zh_CN/faq_series/faq_selected_30.md b/src/PaddleClas/docs/zh_CN/faq_series/faq_selected_30.md new file mode 100644 index 0000000..3382673 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/faq_series/faq_selected_30.md @@ -0,0 +1,242 @@ +# FAQ + +## 写在前面 + +* 我们收集整理了开源以来在 issues 和用户群中的常见问题并且给出了简要解答，旨在为图像分类的开发者提供一些参考，也希望帮助大家少走一些弯路。 + +* 图像分类领域大佬众多，模型和论文更新速度也很快，本文档回答主要依赖有限的项目实践，难免挂一漏万，如有遗漏和不足，也希望有识之士帮忙补充和修正，万分感谢。 + + +## PaddleClas 常见问题汇总 + +* [1. 图像分类 30 个问题](#1) + * [1.1 基础知识](#1.1) + * [1.2 模型训练相关](#1.2) + * [1.3 数据相关](#1.3) + * [1.4 模型推理与预测相关](#1.4) +* [2. PaddleClas 使用问题](#2) + + + +## 1. 图像分类 30 个问题 + + +### 1.1 基础知识 + +>> +* Q: 图像分类领域常用的分类指标有几种 +* A: + * 对于单个标签的图像分类问题（仅包含 1 个类别与背景），评估指标主要有 Accuracy，Precision，Recall，F-score 等，令 TP(True Positive)表示将正类预测为正类，FP(False Positive)表示将负类预测为正类，TN(True Negative)表示将负类预测为负类，FN(False Negative)表示将正类预测为负类。那么 Accuracy=(TP + TN) / NUM，Precision=TP /(TP + FP)，Recall=TP /(TP + FN)。 + * 对于类别数大于 1 的图像分类问题，评估指标主要有 Accuary 和 Class-wise Accuracy，Accuary 表示所有类别预测正确的图像数量占总图像数量的百分比；Class-wise Accuracy 是对每个类别的图像计算 Accuracy，然后再对所有类别的 Accuracy 取平均得到。 + +>> +* Q: 怎样根据自己的任务选择合适的模型进行训练？ +* A: 如果希望在服务器部署，或者希望精度尽可能地高，对模型存储大小或者预测速度的要求不是很高，那么推荐使用 ResNet_vd、Res2Net_vd、DenseNet、Xception 等适合于服务器端的系列模型；如果希望在移动端侧部署，则推荐使用 MobileNetV3、GhostNet 等适合于移动端的系列模型。同时，我们推荐在选择模型的时候可以参考[模型库](../models/models_intro.md)中的速度-精度指标图。 + +>> +* Q: 如何进行参数初始化，什么样的初始化可以加快模型收敛？ +* A: 众所周知，参数的初始化可以影响模型的最终性能。一般来说，如果目标数据集不是很大，建议使用 ImageNet-1k 训练得到的预训练模型进行初始化。如果是自己手动设计的网络或者暂时没有基于 ImageNet-1k 训练得到的预训练权重，可以使用 Xavier 初始化或者 MSRA 初始化，其中 Xavier 初始化是针对 Sigmoid 函数提出的，对 RELU 函数不太友好，网络越深，各层输入的方差越小，网络越难训练，所以当神经网络中使用较多 RELU 激活函数时，推荐使用 MSRA 初始化。 + +>> +* Q: 针对深度神经网络参数冗余的问题，目前有哪些比较好的解决办法？ +* A: 目前有几种主要的方法对模型进行压缩，减少模型参数冗余的问题，如剪枝、量化、知识蒸馏等。模型剪枝指的是将权重矩阵中相对不重要的权值剔除，然后再重新对网络进行微调；模型量化指的是一种将浮点计算转成低比特定点计算的技术，如 8 比特、4 比特等，可以有效的降低模型计算强度、参数大小和内存消耗。知识蒸馏是指使用教师模型(teacher model)去指导学生模型(student model)学习特定任务，保证小模型在参数量不变的情况下，性能有较大的提升，甚至获得与大模型相似的精度指标。 + +>> +* Q: 怎样在其他任务，如目标检测、图像分割、关键点检测等任务中选择比较合适的分类模型作为骨干网络？ +* A: 在不考虑速度的情况下，在大部分的任务中，推荐使用精度更高的预训练模型和骨干网络，PaddleClas 中开源了一系列的 SSLD 知识蒸馏预训练模型，如 ResNet50_vd_ssld, Res2Net200_vd_26w_4s_ssld 等，在模型精度和速度方面都是非常有优势的，推荐大家使用。对于一些特定的任务，如图像分割或者关键点检测等任务，对图像分辨率的要求比较高，那么更推荐使用 HRNet 等能够同时兼顾网络深度和分辨率的神经网络模型，PaddleClas 也提供了 HRNet_W18_C_ssld、HRNet_W48_C_ssld 等精度非常高的 HRNet SSLD 蒸馏系列预训练模型，大家可以使用这些精度更高的预训练模型与骨干网络，提升自己在其他任务上的模型精度。 + +>> +* Q: 注意力机制是什么？目前有哪些比较常用的注意力机制方法？ +* A: 注意力机制(Attention Mechanism)源于对人类视觉的研究。将注意力机制用在计算机视觉任务上，可以有效捕捉图片中有用的区域，从而提升整体网络性能。目前比较常用的有 [SE block](https://arxiv.org/abs/1709.01507)、[SK-block](https://arxiv.org/abs/1903.06586)、[Non-local block](https://arxiv.org/abs/1711.07971)、[GC block](https://arxiv.org/abs/1904.11492)、[CBAM](https://arxiv.org/abs/1807.06521) 等，核心思想就是去学习特征图在不同区域或者不同通道中的重要性，从而让网络更加注意显著性的区域。 + + +### 1.2 模型训练相关 + +>> +* Q: 使用深度卷积网络做图像分类，如果训练一个拥有 1000 万个类的模型会碰到什么问题？ +* A: 因为 FC 层参数很多，内存/显存/模型的存储占用都会大幅增大；模型收敛速度也会变慢一些。建议在这种情况下，再最后的 FC 层前加一层维度较小的 FC，这样可以大幅减少模型的存储大小。 + +>> +* Q: 训练过程中，如果模型收敛效果很差，可能的原因有哪些呢？ +* A: 主要有以下几个可以排查的地方：（1）应该检查数据标注，确保训练集和验证集的数据标注没有问题。（2）可以试着调整一下学习率（初期可以以 10 倍为单位进行调节），过大（训练震荡）或者过小（收敛太慢）的学习率都可能导致收敛效果差。（3）数据量太大，选择的模型太小，难以学习所有数据的特征。（4）可以看下数据预处理的过程中是否使用了归一化，如果没有使用归一化操作，收敛速度可能会比较慢。（5）如果数据量比较小，可以试着加载 PaddleClas 中提供的基于 ImageNet-1k 数据集的预训练模型，这可以大大提升训练收敛速度。（6）数据集存在长尾问题，可以参考[数据长尾问题解决方案](#long_tail)。 + +>> +* Q: 训练图像分类任务时，该怎么选择合适的优化器？ +* A: 优化器的目的是为了让损失函数尽可能的小，从而找到合适的参数来完成某项任务。目前业界主要用到的优化器有 SGD、RMSProp、Adam、AdaDelt 等，其中由于带 momentum 的 SGD 优化器广泛应用于学术界和工业界，所以我们发布的模型也大都使用该优化器来实现损失函数的梯度下降。带 momentum 的 SGD 优化器有两个劣势，其一是收敛速度慢，其二是初始学习率的设置需要依靠大量的经验，然而如果初始学习率设置得当并且迭代轮数充足，该优化器也会在众多的优化器中脱颖而出，使得其在验证集上获得更高的准确率。一些自适应学习率的优化器如 Adam、RMSProp 等，收敛速度往往比较快，但是最终的收敛精度会稍差一些。如果追求更快的收敛速度，我们推荐使用这些自适应学习率的优化器，如果追求更高的收敛精度，我们推荐使用带 momentum 的 SGD 优化器。 + +>> +* Q: 当前主流的学习率下降策略有哪些？一般需要怎么选择呢？ +* A: 学习率是通过损失函数的梯度调整网络权重的超参数的速度。学习率越低，损失函数的变化速度就越慢。虽然使用低学习率可以确保不会错过任何局部极小值，但也意味着将花费更长的时间来进行收敛，特别是在被困在高原区域的情况下。在整个训练过程中，我们不能使用同样的学习率来更新权重，否则无法到达最优点，所以需要在训练过程中调整学习率的大小。在训练初始阶段，由于权重处于随机初始化的状态，损失函数相对容易进行梯度下降，所以可以设置一个较大的学习率。在训练后期，由于权重参数已经接近最优值，较大的学习率无法进一步寻找最优值，所以需要设置一个较小的学习率。在训练整个过程中，很多研究者使用的学习率下降方式是 piecewise_decay，即阶梯式下降学习率，如在 ResNet50 标准的训练中，我们设置的初始学习率是 0.1，每 30 epoch 学习率下降到原来的 1/10，一共迭代 120 epoch。除了 piecewise_decay，很多研究者也提出了学习率的其他下降方式，如 polynomial_decay（多项式下降）、exponential_decay（指数下降）, cosine_decay（余弦下降）等，其中 cosine_decay 无需调整超参数，鲁棒性也比较高，所以成为现在提高模型精度首选的学习率下降方式。Cosine_decay 和 piecewise_decay 的学习率变化曲线如下图所示，容易观察到，在整个训练过程中，cosine_decay 都保持着较大的学习率，所以其收敛较为缓慢，但是最终的收敛效果较 peicewise_decay 更好一些。 +![](../../images/models/lr_decay.jpeg) +>> +* Q: Warmup 学习率策略是什么？一般用在什么样的场景中？ +* A: Warmup 策略顾名思义就是让学习率先预热一下，在训练初期我们不直接使用最大的学习率，而是用一个逐渐增大的学习率去训练网络，当学习率增大到最高点时，再使用学习率下降策略中提到的学习率下降方式衰减学习率的值。如果使用较大的 batch_size 训练神经网络时，我们建议您使用 warmup 策略。实验表明，在 batch_size 较大时，warmup 可以稳定提升模型的精度。在训练 MobileNetV3 等 batch_size 较大的实验中，我们默认将 warmup 中的 epoch 设置为 5，即先用 5 epoch 将学习率从 0 增加到最大值，再去做相应的学习率衰减。 + +>> +* Q: 什么是 `batch size`？在模型训练中，怎么选择合适的 `batch size`？ +* A: `batch size` 是训练神经网络中的一个重要的超参数，该值决定了一次将多少数据送入神经网络参与训练。论文 [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://arxiv.org/abs/1706.02677)，当 `batch size` 的值与学习率的值呈线性关系时，收敛精度几乎不受影响。在训练 ImageNet 数据时，大部分的神经网络选择的初始学习率为 0.1，`batch size` 是 256，所以根据实际的模型大小和显存情况，可以将学习率设置为 0.1*k, batch_size 设置为 256*k。在实际任务中，也可以将该设置作为初始参数，进一步调节学习率参数并获得更优的性能。 +>> +* Q: weight_decay 是什么？怎么选择合适的 weight_decay 呢？ +* A: 过拟合是机器学习中常见的一个名词，简单理解即为模型在训练数据上表现很好，但在测试数据上表现较差，在卷积神经网络中，同样存在过拟合的问题，为了避免过拟合，很多正则方式被提出，其中，weight_decay 是其中一个广泛使用的避免过拟合的方式。在使用 SGD 优化器时，weight_decay 等价于在最终的损失函数后添加 L2 正则化，L2 正则化使得网络的权重倾向于选择更小的值，最终整个网络中的参数值更趋向于 0，模型的泛化性能相应提高。在各大深度学习框架的实现中，该值表达的含义是 L2 正则前的系数，在 paddle 框架中，该值的名称是 l2_decay，所以以下都称其为 l2_decay。该系数越大，表示加入的正则越强，模型越趋于欠拟合状态。在训练 ImageNet 的任务中，大多数的网络将该参数值设置为 1e-4，在一些小的网络如 MobileNet 系列网络中，为了避免网络欠拟合，该值设置为 1e-5~4e-5 之间。当然，该值的设置也和具体的数据集有关系，当任务的数据集较大时，网络本身趋向于欠拟合状态，可以将该值适当减小，当任务的数据集较小时，网络本身趋向于过拟合状态，可以将该值适当增大。下表展示了 MobileNetV1_x0_25 在 ImageNet-1k 上使用不同 l2_decay 的精度情况。由于 MobileNetV1_x0_25 是一个比较小的网络，所以 l2_decay 过大会使网络趋向于欠拟合状态，所以在该网络中，相对 1e-4，3e-5 是更好的选择。 + +| 模型 | L2_decay | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV1_x0_25 | 1e-4 | 43.79%/67.61% | 50.41%/74.70% | +| MobileNetV1_x0_25 | 3e-5 | 47.38%/70.83% | 51.45%/75.45% | + + +>> +* Q: 标签平滑(label_smoothing)指的是什么？有什么效果呢？一般适用于什么样的场景中？ +* A: Label_smoothing 是深度学习中的一种正则化方法，其全称是 Label Smoothing Regularization(LSR)，即标签平滑正则化。在传统的分类任务计算损失函数时，是将真实的 one hot 标签与神经网络的输出做相应的交叉熵计算，而 label_smoothing 是将真实的 one hot 标签做一个标签平滑的处理，使得网络学习的标签不再是一个 hard label，而是一个有概率值的 soft label，其中在类别对应的位置的概率最大，其他位置概率是一个非常小的数。具体的计算方式参见论文[2]。在 label_smoothing 里，有一个 epsilon 的参数值，该值描述了将标签软化的程度，该值越大，经过 label smoothing 后的标签向量的标签概率值越小，标签越平滑，反之，标签越趋向于 hard label，在训练 ImageNet-1k 的实验里通常将该值设置为 0.1。 +在训练 ImageNet-1k 的实验中，我们发现，ResNet50 大小级别及其以上的模型在使用 label_smooting 后，精度有稳定的提升。下表展示了 ResNet50_vd 在使用 label_smoothing 前后的精度指标。同时，由于 label_smoohing 相当于一种正则方式，在相对较小的模型上，精度提升不明显甚至会有所下降，下表展示了 ResNet18 在 ImageNet-1k 上使用 label_smoothing 前后的精度指标。可以明显看到，在使用 label_smoothing 后，精度有所下降。 + +| 模型 | Use_label_smoothing | Test acc1 | +|:--:|:--:|:--:| +| ResNet50_vd | 0 | 77.9% | +| ResNet50_vd | 1 | 78.4% | +| ResNet18 | 0 | 71.0% | +| ResNet18 | 1 | 70.8% | + + +>> +* Q: 在训练的时候怎么通过训练集和验证集的准确率或者 loss 确定进一步的调优策略呢？ +* A: 在训练网络的过程中，通常会打印每一个 epoch 的训练集准确率和验证集准确率，二者刻画了该模型在两个数据集上的表现。通常来说，训练集的准确率比验证集准确率微高或者二者相当是比较不错的状态。如果发现训练集的准确率比验证集高很多，说明在这个任务上已经过拟合，需要在训练过程中加入更多的正则，如增大 l2_decay 的值，加入更多的数据增广策略，加入 label_smoothing 策略等；如果发现训练集的准确率比验证集低一些，说明在这个任务上可能欠拟合，需要在训练过程中减弱正则效果，如减小 l2_decay 的值，减少数据增广方式，增大图片 crop 区域面积，减弱图片拉伸变换，去除 label_smoothing 等。 + +>> +* Q: 怎么使用已有的预训练模型提升自己的数据集的精度呢？ +* A: 在现阶段计算机视觉领域中，加载预训练模型来训练自己的任务已成为普遍的做法，相比从随机初始化开始训练，加载预训练模型往往可以提升特定任务的精度。一般来说，业界广泛使用的预训练模型是通过训练 128 万张图片 1000 类的 ImageNet-1k 数据集得到的，该预训练模型的 fc 层权重是是一个 k\*1000 的矩阵，其中 k 是 fc 层以前的神经元数，在加载预训练权重时，无需加载 fc 层的权重。在学习率方面，如果您的任务训练的数据集特别小（如小于 1 千张），我们建议你使用较小的初始学习率，如 0.001（batch_size:256,下同），以免较大的学习率破坏预训练权重。如果您的训练数据集规模相对较大（大于 10 万），我们建议你尝试更大的初始学习率，如 0.01 或者更大。 + + +### 1.3 数据相关 + +>> +* Q: 图像分类的数据预处理过程一般包括哪些步骤？ +* A: 以在 ImageNet-1k 数据集上训练 ResNet50 为例，一张图片被输入进网络，主要有图像解码、随机裁剪、随机水平翻转、标准化、数据重排，组 batch 并送进网络这几个步骤。图像解码指的是将图片文件读入到内存中，随机裁剪指的是将读入的图像随机拉伸并裁剪到长宽均为 224 的图像，随机水平翻转指的是对裁剪后的图片以 0.5 的概率进行水平翻转，标准化指的是将图片每个通道的数据通过去均值实现中心化的处理，使得数据尽可能符合 `N(0,1)` 的正态分布，数据重排指的是将数据由 `[224,224,3]` 的格式变为 `[3,224,224]` 的格式，组 batch 指的是将多幅图像组成一个批数据，送进网络进行训练。 + +>> +* Q: 随机裁剪是怎么影响小模型训练的性能的？ +* A: 在 ImageNet-1k 数据的标准预处理中，随机裁剪函数中定义了 scale 和 ratio 两个值，两个值分别确定了图片 crop 的大小和图片的拉伸程度，其中 scale 的默认取值范围是 0.08-1(lower_scale-upper_scale),ratio 的默认取值范围是 3/4-4/3(lower_ratio-upper_ratio)。在非常小的网络训练中，此类数据增强会使得网络欠拟合，导致精度有所下降。为了提升网络的精度，可以使其数据增强变的更弱，即增大图片的 crop 区域或者减弱图片的拉伸变换程度。我们可以分别通过增大 lower_scale 的值或缩小 lower_ratio 与 upper_scale 的差距来实现更弱的图片变换。下表列出了使用不同 lower_scale 训练 MobileNetV2_x0_25 的精度，可以看到，增大图片的 crop 区域面积后训练精度和验证精度均有提升。 + +| 模型 | Scale 取值范围 | Train_acc1/acc5 | Test_acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV2_x0_25 | [0.08,1] | 50.36%/72.98% | 52.35%/75.65% | +| MobileNetV2_x0_25 | [0.2,1] | 54.39%/77.08% | 53.18%/76.14% | + + +>> +* Q: 数据量不足的情况下，目前有哪些常见的数据增广方法来增加训练样本的丰富度呢？ +* A: PaddleClas 中将目前比较常见的数据增广方法分为了三大类，分别是图像变换类、图像裁剪类和图像混叠类，图像变换类主要包括 AutoAugment 和 RandAugment，图像裁剪类主要包括 CutOut、RandErasing、HideAndSeek 和 GridMask，图像混叠类主要包括 Mixup 和 Cutmix，更详细的关于数据增广的介绍可以参考：[数据增广章节](../algorithm_introduction/DataAugmentation.md)。 +>> +* Q: 对于遮挡情况比较常见的图像分类场景，该使用什么数据增广方法去提升模型的精度呢？ +* A: 在训练的过程中可以尝试对训练集使用 CutOut、RandErasing、HideAndSeek 和 GridMask 等裁剪类数据增广方法，让模型也能够不止学习到显著区域，也能关注到非显著性区域，从而在遮挡的情况下，也能较好地完成识别任务。 + +>> +* Q: 对于色彩变换情况比较复杂的情况下，应该使用哪些数据增广方法提升模型精度呢？ +* A: 可以考虑使用 AutoAugment 或者 RandAugment 的数据增广策略，这两种策略中都包括了锐化、直方图均衡化等丰富的颜色变换，可以让模型在训练的过程中对这些变换更加鲁棒。 +>> +* Q: Mixup 和 Cutmix 的工作原理是什么？为什么它们也是非常有效的数据增广方法？ +* A: Mixup 通过线性叠加两张图片生成新的图片，对应 label 也进行线性叠加用以训练，Cutmix 则是从一幅图中随机裁剪出一个 感兴趣区域(ROI)，然后覆盖当前图像中对应的区域，label 也按照图像面积比例进行线性叠加。它们其实也是生成了和训练集不同的样本和 label 并让网络去学习，从而扩充了样本的丰富度。 +>> +* Q: 对于精度要求不是那么高的图像分类任务，大概需要准备多大的训练数据集呢？ +* A: 训练数据的数量和需要解决问题的复杂度有关系。难度越大，精度要求越高，则数据集需求越大，而且一般情况实际中的训练数据越多效果越好。当然，一般情况下，在加载预训练模型的情况下，每个类别包括 10-20 张图像即可保证基本的分类效果；不加载预训练模型的情况下，每个类别需要至少包含 100-200 张图像以保证基本的分类效果。 + + +>> +* Q: 对于长尾分布的数据集，目前有哪些比较常用的方法？ +* A:（1）可以对数据量比较少的类别进行重采样，增加其出现的概率；（2）可以修改 loss，增加图像较少对应的类别的图片的 loss 权重；（3）可以借鉴迁移学习的方法，从常见类别中学习通用知识，然后迁移到少样本的类别中。 + + +### 1.4 模型推理与预测相关 + +>> +* Q: 有时候图像中只有小部分区域是所关注的前景物体，直接拿原图来进行分类的话，识别效果很差，这种情况要怎么做呢？ +* A: 可以在分类之前先加一个主体检测的模型，将前景物体检测出来之后再进行分类，可以大大提升最终的识别效果。如果不考虑时间成本，也可以使用 multi-crop 的方式对所有的预测做融合来决定最终的类别。 +>> +* Q: 目前推荐的，模型预测方式有哪些？ +* A: 在模型训练完成之后，推荐使用导出的固化模型（inference model），基于 Paddle 预测引擎进行预测，目前支持 python inference 与 cpp inference。如果希望基于服务化部署预测模型，那么推荐使用 PaddleServing 的部署方式。 +>> +* Q: 模型训练完成之后，有哪些比较合适的预测方法进一步提升模型精度呢？ +* A:（1）可以使用更大的预测尺度，比如说训练的时候使用的是 224，那么预测的时候可以考虑使用 288 或者 320，这会直接带来 0.5% 左右的精度提升。（2）可以使用测试时增广的策略（Test Time Augmentation, TTA)，将测试集通过旋转、翻转、颜色变换等策略，创建多个副本，并分别预测，最后将所有的预测结果进行融合，这可以大大提升预测结果的精度和鲁棒性。（3）当然，也可以使用多模型融合的策略，将多个模型针对相同图片的预测结果进行融合。 +>> +* Q: 多模型融合的时候，该怎么选择合适的模型进行融合呢？ +* A: 在不考虑预测速度的情况下，建议选择精度尽量高的模型；同时建议选择不同结构或者系列的模型进行融合，比如在精度相似的情况下，ResNet50_vd 与 Xception65 的模型融合结果往往比 ResNet50_vd 与 ResNet101_vd 的模型融合结果要好一些。 + +>> +* Q: 使用固定的模型进行预测时有哪些比较常用的加速方法？ +* A:（1）使用性能更优的 GPU 进行预测；（2）增大预测的 batch size；（3）使用 TenorRT 以及 FP16 半精度浮点数等方法进行预测。 + + + +## 2. PaddleClas 使用问题 + +>> +* Q: 评估和预测时，已经指定了预训练模型所在文件夹的地址，但是仍然无法导入参数，这么为什么呢？ +* A: 加载预训练模型时，需要指定预训练模型的前缀，例如预训练模型参数所在的文件夹为 `output/ResNet50_vd/19`，预训练模型参数的名称为 `output/ResNet50_vd/19/ppcls.pdparams`，则 `pretrained_model` 参数需要指定为 `output/ResNet50_vd/19/ppcls`，PaddleClas 会自动补齐`.pdparams` 的后缀。 + + +>> +* Q: 在评测 `EfficientNetB0_small` 模型时，为什么最终的精度始终比官网的低 0.3% 左右？ +* A: `EfficientNet` 系列的网络在进行 resize 的时候，是使用 `cubic 插值方式`(resize 参数的 interpolation 值设置为 2)，而其他模型默认情况下为 None，因此在训练和评估的时候需要显式地指定 resize 的 interpolation 值。具体地，可以参考以下配置中预处理过程中 ResizeImage 的参数。 +``` + Eval: + dataset: + name: ImageNetDataset + image_root: ./dataset/ILSVRC2012/ + cls_label_path: ./dataset/ILSVRC2012/val_list.txt + transform_ops: + - DecodeImage: + to_rgb: True + channel_first: False + - ResizeImage: + resize_short: 256 + interpolation: 2 + - CropImage: + size: 224 + - NormalizeImage: + scale: 1.0/255.0 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' +``` + +>> +* Q: python2 下，使用 visualdl 的时候，报出以下错误，`TypeError: __init__() missing 1 required positional argument: 'sync_cycle'`，这是为什么呢？ +* A: 目前 visualdl 仅支持在 python3 下运行，visualdl 需要是 2.0 以上的版本，如果 visualdl 版本不对的话，可以通过以下方式进行安装：`pip3 install visualdl -i https://mirror.baidu.com/pypi/simple` + +>> +* Q: 自己在测 ResNet50_vd 预测单张图片速度的时候发现比官网提供的速度 benchmark 慢了很多，而且 CPU 速度比 GPU 速度快很多，这个是为什么呢？ +* A: 模型预测需要初始化，初始化的过程比较耗时，因此在统计预测速度的时候，需要批量跑一批图片，去除前若干张图片的预测耗时，再统计下平均的时间。GPU 比 CPU 速度测试单张图片速度慢是因为 GPU 的初始化并 CPU 要慢很多。 + +>> +* Q: 灰度图可以用于模型训练吗？ +* A: 灰度图也可以用于模型训练，不过需要修改模型的输入 shape 为 `[1, 224, 224]`，此外数据增广部分也需要注意适配一下。不过为了更好地使用 PaddleClas 代码的话，即使是灰度图，也建议调整为 3 通道的图片进行训练（RGB 通道的像素值相等）。 + +>> +* Q: 怎么在 windows 上或者 cpu 上面模型训练呢？ +* A: 可以参考[开始使用教程](../models_training/classification.md)，详细介绍了在 Linux、Windows、CPU 等环境中进行模型训练、评估与预测的教程。 +>> +* Q: 怎样在模型训练的时候使用 label smoothing 呢？ +* A: 可以在配置文件中的 `Loss` 字段下进行设置，如下所示，`epsilon=0.1` 表示设置该值为 0.1，若不设置 `epsilon` 字段，则不使用 `label smoothing`。 +```yaml +Loss: + Train: + - CELoss: + weight: 1.0 + epsilon: 0.1 +``` +>> +* Q: PaddleClas 提供的 10W 类图像分类预训练模型能否用于模型推断呢？ +* A: 该 10W 类图像分类预训练模型没有提供 fc 全连接层的参数，无法用于模型推断，目前可以用于模型微调。 +>> +* Q: 在使用 `deploy/python/predict_cls.py` 进行模型预测的时候，报了这个问题: `Error: Pass tensorrt_subgraph_pass has not been registered`，这是为什么呢？ +* A: 如果希望使用 TensorRT 进行模型预测推理的话，需要安装或是自己编译带 TensorRT 的 PaddlePaddle，Linux、Windows、macOS 系统的用户下载安装可以参考参考[下载预测库](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html)，如果没有符合您所需要的版本，则需要本地编译安装，编译方法可以参考[源码编译](https://paddleinference.paddlepaddle.org.cn/user_guides/source_compile.html)。 +>> +* Q: 怎样在训练的时候使用自动混合精度(Automatic Mixed Precision, AMP)训练呢？ +* A: 可以参考 [ResNet50_fp16.yaml](../../../ppcls/configs/ImageNet/ResNet/ResNet50_fp16.yaml) 这个配置文件；具体地，如果希望自己的配置文件在模型训练的时候也支持自动混合精度，可以在配置文件中添加下面的配置信息。 +``` +# mixed precision training +AMP: + scale_loss: 128.0 + use_dynamic_loss_scaling: True + use_pure_fp16: &use_pure_fp16 True +``` diff --git a/src/PaddleClas/docs/zh_CN/faq_series/index.rst b/src/PaddleClas/docs/zh_CN/faq_series/index.rst new file mode 100644 index 0000000..225b82d --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/faq_series/index.rst @@ -0,0 +1,10 @@ +FAQ系列 +================================ + +.. toctree:: + :maxdepth: 2 + + faq_2021_s2.md + faq_2021_s1.md + faq_2020_s1.md + faq_selected_30.md diff --git a/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/feature_extraction.md b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/feature_extraction.md new file mode 100644 index 0000000..1438e96 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/feature_extraction.md @@ -0,0 +1,182 @@ +# 特征提取 + +## 目录 + +- [1. 简介](#1) +- [2. 网络结构](#2) +- [3. 通用识别模型](#3) +- [4. 自定义特征提取](#4) + - [4.1 数据准备](#4.1) + - [4.2 模型训练](#4.2) + - [4.3 模型评估](#4.3) + - [4.4 模型推理](#4.4) + - [4.4.1 导出推理模型](#4.4.1) + - [4.4.2 获取特征向量](#4.4.2) + + + +## 1. 简介 + +特征提取是图像识别中的关键一环，它的作用是将输入的图片转化为固定维度的特征向量，用于后续的[向量检索](./vector_search.md)。好的特征需要具备相似度保持性，即在特征空间中，相似度高的图片对其特征相似度要比较高（距离比较近），相似度低的图片对，其特征相似度要比较小（距离比较远）。[Deep Metric Learning](../algorithm_introduction/metric_learning.md)用以研究如何通过深度学习的方法获得具有强表征能力的特征。 + + + +## 2. 网络结构 +为了图像识别任务的灵活定制，我们将整个网络分为 Backbone、 Neck、 Head 以及 Loss 部分，整体结构如下图所示: +![](../../images/feature_extraction_framework.png) +图中各个模块的功能为: + +- **Backbone**: 指定所使用的骨干网络。 值得注意的是，PaddleClas 提供的基于 ImageNet 的预训练模型，最后一层的输出为 1000，我们需要依据所需的特征维度定制最后一层的输出。 +- **Neck**: 用以特征增强及特征维度变换。这儿的 Neck，可以是一个简单的 Linear Layer，用来做特征维度变换；也可以是较复杂的 FPN 结构，用以做特征增强。 +- **Head**: 用来将 feature 转化为 logits。除了常用的 Fc Layer 外，还可以替换为 cosmargin, arcmargin, circlemargin 等模块。 +- **Loss**: 指定所使用的 Loss 函数。我们将 Loss 设计为组合 loss 的形式，可以方便地将 Classification Loss 和 Pair_wise Loss 组合在一起。 + + + +## 3. 通用识别模型 + +在 PP-Shitu 中, 我们采用 [PP_LCNet_x2_5](../models/PP-LCNet.md) 作为骨干网络 Neck 部分选用 Linear Layer, Head 部分选用 [ArcMargin](../../../ppcls/arch/gears/arcmargin.py)，Loss 部分选用 CELoss，详细的配置文件见[通用识别配置文件](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml)。其中，训练数据为如下 7 个公开数据集的汇总： + +| 数据集 | 数据量 | 类别数 | 场景 | 数据集地址 | +| :------------: | :-------------: | :-------: | :-------: | :--------: | +| Aliproduct | 2498771 | 50030 | 商品 | [地址](https://retailvisionworkshop.github.io/recognition_challenge_2020/) | +| GLDv2 | 1580470 | 81313 | 地标 | [地址](https://github.com/cvdfoundation/google-landmark) | +| VeRI-Wild | 277797 | 30671 | 车辆 | [地址](https://github.com/PKU-IMRE/VERI-Wild)| +| LogoDet-3K | 155427 | 3000 | Logo | [地址](https://github.com/Wangjing1551/LogoDet-3K-Dataset) | +| iCartoonFace | 389678 | 5013 | 动漫人物 | [地址](http://challenge.ai.iqiyi.com/detail?raceId=5def69ace9fcf68aef76a75d) | +| SOP | 59551 | 11318 | 商品 | [地址](https://cvgl.stanford.edu/projects/lifted_struct/) | +| Inshop | 25882 | 3997 | 商品 | [地址](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html) | +| **Total** | **5M** | **185K** | ---- | ---- | + +最终的模型效果如下表所示: + +| 模型 | Aliproduct | VeRI-Wild | LogoDet-3K | iCartoonFace | SOP | Inshop | Latency(ms) | +| :----------: | :---------: | :-------: | :-------: | :--------: | :--------: | :--------: | :--------: | +PP-LCNet-2.5x | 0.839 | 0.888 | 0.861 | 0.841 | 0.793 | 0.892 | 5.0 + +* 采用的评测指标为：`Recall@1` +* 速度评测机器的 CPU 具体信息为：`Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz` +* 速度指标的评测条件为： 开启 MKLDNN, 线程数设置为 10 +* 预训练模型地址：[通用识别预训练模型](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/general_PPLCNet_x2_5_pretrained_v1.0.pdparams) + + + +## 4. 自定义特征提取 + +自定义特征提取，是指依据自己的任务，重新训练特征提取模型。主要包含四个步骤：1）数据准备；2）模型训练；3）模型评估；4）模型推理。 + + + +### 4.1 数据准备 + +首先，需要基于任务定制自己的数据集。数据集格式参见[格式说明](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/zh_CN/data_preparation/recognition_dataset.md#%E6%95%B0%E6%8D%AE%E9%9B%86%E6%A0%BC%E5%BC%8F%E8%AF%B4%E6%98%8E)。在启动模型训练之前，需要在配置文件中修改数据配置相关的内容, 主要包括数据集的地址以及类别数量。对应到配置文件中的位置如下所示： +``` + Head: + name: ArcMargin + embedding_size: 512 + class_num: 185341 #此处表示类别数 +``` +``` + Train: + dataset: + name: ImageNetDataset + image_root: ./dataset/ #此处表示train数据所在的目录 + cls_label_path: ./dataset/train_reg_all_data.txt #此处表示train数据集label文件的地址 +``` +``` + Query: + dataset: + name: VeriWild + image_root: ./dataset/Aliproduct/. #此处表示query数据集所在的目录 + cls_label_path: ./dataset/Aliproduct/val_list.txt. #此处表示query数据集label文件的地址 +``` +``` + Gallery: + dataset: + name: VeriWild + image_root: ./dataset/Aliproduct/ #此处表示gallery数据集所在的目录 + cls_label_path: ./dataset/Aliproduct/val_list.txt. #此处表示gallery数据集label文件的地址 +``` + + + +### 4.2 模型训练 + +- 单机单卡训练 +```shell +export CUDA_VISIBLE_DEVICES=0 +python tools/train.py -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml +``` +- 单机多卡训练 +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" tools/train.py \ + -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml +``` +**注意:** +配置文件中默认采用`在线评估`的方式，如果你想加快训练速度，去除`在线评估`，只需要在上述命令后面，增加 `-o eval_during_train=False`。训练完毕后，在 output 目录下会生成最终模型文件 `latest`，`best_model` 和训练日志文件 `train.log`。其中，`best_model` 用来存储当前评测指标下的最佳模型；`latest` 用来存储最新生成的模型, 方便在任务中断的情况下从断点位置启动训练。 + +- 断点续训： +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" tools/train.py \ + -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ + -o Global.checkpoint="output/RecModel/latest" +``` + + + +### 4.3 模型评估 + +- 单卡评估 +```shell +export CUDA_VISIBLE_DEVICES=0 +python tools/eval.py \ +-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ +-o Global.pretrained_model="output/RecModel/best_model" +``` + +- 多卡评估 +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" tools/eval.py \ + -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ + -o Global.pretrained_model="output/RecModel/best_model" +``` +**推荐：** 建议使用多卡评估。多卡评估方式可以利用多卡并行计算快速得到整体数据集的特征集合，能够加速评估的过程。 + + + +### 4.4 模型推理 + +推理过程包括两个步骤： 1)导出推理模型; 2)获取特征向量 + + + +#### 4.4.1 导出推理模型 + +``` +python tools/export_model.py \ +-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \ +-o Global.pretrained_model="output/RecModel/best_model" +``` +生成的推理模型位于 `inference` 目录，里面包含三个文件，分别为 `inference.pdmodel`、`inference.pdiparams`、`inference.pdiparams.info`。 +其中: `inference.pdmodel` 用来存储推理模型的结构, `inference.pdiparams` 和 `inference.pdiparams.info` 用来存储推理模型相关的参数信息。 + + + +#### 4.4.2 获取特征向量 + +``` +cd deploy +python python/predict_rec.py \ +-c configs/inference_rec.yaml \ +-o Global.rec_inference_model_dir="../inference" +``` +得到的特征输出格式如下图所示： +![](../../images/feature_extraction_output.png) + +在实际使用过程中，单纯得到特征往往并不能够满足业务的需求。如果想进一步通过特征检索来进行图像识别，可以参照文档[向量检索](./vector_search.md)。 diff --git a/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/index.rst b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/index.rst new file mode 100644 index 0000000..6cf5e5a --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/index.rst @@ -0,0 +1,9 @@ +图像识别流程 +================================ + +.. toctree:: + :maxdepth: 2 + + mainbody_detection.md + feature_extraction.md + vector_search.md diff --git a/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/mainbody_detection.md b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/mainbody_detection.md new file mode 100644 index 0000000..f3d7989 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/mainbody_detection.md @@ -0,0 +1,248 @@ +# 主体检测 + + +主体检测技术是目前应用非常广泛的一种检测技术，它指的是检测出图片中一个或者多个主体的坐标位置，然后将图像中的对应区域裁剪下来，进行识别，从而完成整个识别过程。主体检测是识别任务的前序步骤，可以有效提升识别精度。 + +本部分主要从数据集、模型选择和模型训练 3 个方面对该部分内容进行介绍。 + +---------- + +## 目录 + +- [1. 数据集](#1) +- [2. 模型选择](#2) + - [2.1 轻量级主体检测模型](#2.1) + - [2.2 服务端主体检测模型](#2.2) +- [3. 模型训练](#3) + - [3.1 环境准备](#3.1) + - [3.2 数据准备](#3.2) + - [3.3 配置文件改动和说明](#3.3) + - [3.4 启动训练](#3.4) + - [3.5 模型预测与调试](#3.5) + - [3.6 模型导出与预测部署](#3.6) + + + +## 1. 数据集 + +在 PaddleClas 的识别任务中，训练主体检测模型时主要用到了以下几个数据集。 + +| 数据集 | 数据量 | 主体检测任务中使用的数据量 | 场景 | 数据集地址 | +| ------------ | ------ | -------------------------- | ------------ | ---------------------------------------------------------- | +| Objects365 | 170W | 6k | 通用场景 | [地址](https://www.objects365.org/overview.html) | +| COCO2017 | 12W | 5k | 通用场景 | [地址](https://cocodataset.org/) | +| iCartoonFace | 2k | 2k | 动漫人脸检测 | [地址](https://github.com/luxiangju-PersonAI/iCartoonFace) | +| LogoDet-3k | 3k | 2k | Logo 检测 | [地址](https://github.com/Wangjing1551/LogoDet-3K-Dataset) | +| RPC | 3k | 3k | 商品检测 | [地址](https://rpc-dataset.github.io/) | + +在实际训练的过程中，将所有数据集混合在一起。由于是主体检测，这里将所有标注出的检测框对应的类别都修改为 `前景` 的类别，最终融合的数据集中只包含 1 个类别，即前景。 + + + +## 2. 模型选择 + +目标检测方法种类繁多，比较常用的有两阶段检测器（如 FasterRCNN 系列等）；单阶段检测器（如 YOLO、SSD 等）；anchor-free 检测器（如 PicoDet、FCOS 等）。PaddleDetection 中针对服务端使用场景，自研了 PP-YOLO 系列模型；针对端侧（CPU 和移动端等）使用场景，自研了 PicoDet 系列模型，在服务端和端侧均处于业界较为领先的水平。 + +基于上述研究，PaddleClas 中提供了 2 个通用主体检测模型，为轻量级与服务端主体检测模型，分别适用于端侧场景以及服务端场景。下面的表格中给出了在上述 5 个数据集上的平均 mAP 以及它们的模型大小、预测速度对比信息。 + +| 模型 | 模型结构 | 预训练模型下载地址 | inference 模型下载地址 | mAP | inference 模型大小(MB) | 单张图片预测耗时(不包含预处理)(ms) | +| ------------------ | --------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ----- | ---------------------- | ---------------------------------- | +| 轻量级主体检测模型 | PicoDet | [地址](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_pretrained.pdparams) | [tar 格式文件地址](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar) [zip 格式文件地址](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.zip) | 40.1% | 30.1 | 29.8 | +| 服务端主体检测模型 | PP-YOLOv2 | [地址](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/ppyolov2_r50vd_dcn_mainbody_v1.0_pretrained.pdparams) | [tar 格式文件地址](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar) [zip 格式文件地址](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.zip) | 42.5% | 210.5 | 466.6 | + +* 注意 + * 由于部分解压缩软件在解压上述 `tar` 格式文件时存在问题，建议非命令行用户下载 `zip` 格式文件并解压。`tar` 格式文件建议使用命令 `tar xf xxx.tar` 解压。 + * 速度评测机器的 CPU 具体信息为：`Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz`，速度指标为开启 mkldnn，线程数设置为 10 测试得到。 + * 主体检测的预处理过程较为耗时，平均每张图在上述机器上的时间在 40~55 ms 左右，没有包含在上述的预测耗时统计中。 + + + +### 2.1 轻量级主体检测模型 + +PicoDet 由 [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection) 提出，是一个适用于 CPU 或者移动端场景的目标检测算法。具体地，它融合了下面一系列优化算法。 + +- [ATSS](https://arxiv.org/abs/1912.02424) +- [Generalized Focal Loss](https://arxiv.org/abs/2006.04388) +- 余弦学习率策略 +- Cycle-EMA +- 轻量级检测 head + + +更多关于 PicoDet 的优化细节与 benchmark 可以参考 [PicoDet 系列模型介绍](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/picodet/README.md)。 + +在轻量级主体检测任务中，为了更好地兼顾检测速度与效果，我们使用 PPLCNet_x2_5 作为主体检测模型的骨干网络，同时将训练与预测的图像尺度修改为了 640x640，其余配置与 [picodet_lcnet_1_5x_416_coco.yml](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/picodet/more_config/picodet_lcnet_1_5x_416_coco.yml) 完全一致。将数据集更换为自定义的主体检测数据集，进行训练，最终得到检测模型。 + + + +### 2.2 服务端主体检测模型 + +PP-YOLO 由 [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection) 提出，从骨干网络、数据增广、正则化策略、损失函数、后处理等多个角度对 yolov3 模型进行深度优化，最终在“速度-精度”方面达到了业界领先的水平。具体地，优化的策略如下。 + +- 更优的骨干网络: ResNet50vd-DCN +- 更大的训练 batch size: 8 GPUs，每 GPU batch_size=24，对应调整学习率和迭代轮数 +- [Drop Block](https://arxiv.org/abs/1810.12890) +- [Exponential Moving Average](https://www.investopedia.com/terms/e/ema.asp) +- [IoU Loss](https://arxiv.org/pdf/1902.09630.pdf) +- [Grid Sensitive](https://arxiv.org/abs/2004.10934) +- [Matrix NMS](https://arxiv.org/pdf/2003.10152.pdf) +- [CoordConv](https://arxiv.org/abs/1807.03247) +- [Spatial Pyramid Pooling](https://arxiv.org/abs/1406.4729) +- 更优的预训练模型 + +更多关于 PP-YOLO 的详细介绍可以参考：[PP-YOLO 模型](https://github.com/PaddlePaddle/PaddleDetection/blob/release%2F2.1/configs/ppyolo/README_cn.md)。 + +在服务端主体检测任务中，为了保证检测效果，我们使用 ResNet50vd-DCN 作为检测模型的骨干网络，使用配置文件 [ppyolov2_r50vd_dcn_365e_coco.yml](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml)，更换为自定义的主体检测数据集，进行训练，最终得到检测模型。 + + + +## 3. 模型训练 + +本节主要介绍怎样基于 PaddleDetection，基于自己的数据集，训练主体检测模型。 + + + +### 3.1 环境准备 + +下载 PaddleDetection 代码，安装 requirements。 + +```shell +cd +git clone https://github.com/PaddlePaddle/PaddleDetection.git + +cd PaddleDetection +# 安装其他依赖 +pip install -r requirements.txt +``` + +更多安装教程，请参考: [安装文档](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/docs/tutorials/INSTALL_cn.md) + + + +### 3.2 数据准备 + +对于自定义数据集，首先需要将自己的数据集修改为 COCO 格式，可以参考[自定义检测数据集教程](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/static/docs/tutorials/Custom_DataSet.md)制作 COCO 格式的数据集。 + +主体检测任务中，所有的检测框均属于前景，在这里需要将标注文件中，检测框的 `category_id` 修改为 1，同时将整个标注文件中的 `categories` 映射表修改为下面的格式，即整个类别映射表中只包含`前景`类别。 + +```json +[{u'id': 1, u'name': u'foreground', u'supercategory': u'foreground'}] +``` + + + +### 3.3 配置文件改动和说明 + +我们使用 `configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml` 配置进行训练，配置文件摘要如下： + +![](../../images/det/PaddleDetection_config.png) + +从上图看到 `ppyolov2_r50vd_dcn_365e_coco.yml` 配置需要依赖其他的配置文件，这些配置文件的含义如下: + +``` +coco_detection.yml：主要说明了训练数据和验证数据的路径 + +runtime.yml：主要说明了公共的运行参数，比如是否使用 GPU、每多少个 epoch 存储 checkpoint 等 + +optimizer_365e.yml：主要说明了学习率和优化器的配置 + +ppyolov2_r50vd_dcn.yml：主要说明模型和主干网络的情况 + +ppyolov2_reader.yml：主要说明数据读取器配置，如 batch size，并发加载子进程数等，同时包含读取后预处理操作，如 resize、数据增强等等 +``` + +在主体检测任务中，需要将 `datasets/coco_detection.yml` 中的 `num_classes` 参数修改为 1（只有 1 个前景类别），同时将训练集和测试集的路径修改为自定义数据集的路径。 + +此外，也可以根据实际情况，修改上述文件，比如，如果显存溢出，可以将 batch size 和学习率等比缩小等。 + + + +### 3.4 启动训练 + +PaddleDetection 提供了单卡/多卡训练模式，满足用户多种训练需求。 + +* GPU 单卡训练 + +```bash +# windows 和 Mac 下不需要执行该命令 +export CUDA_VISIBLE_DEVICES=0 +python tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml +``` + +* GPU 多卡训练 + +```bash +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --eval +``` + +--eval：表示边训练边验证。 + + +* （**推荐**）模型微调 +如果希望加载 PaddleClas 中已经训练好的主体检测模型，在自己的数据集上进行模型微调，可以使用下面的命令进行训练。 + +```bash +export CUDA_VISIBLE_DEVICES=0 +# 指定 pretrain_weights 参数，加载通用的主体检测预训练模型 +python tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml -o pretrain_weights=https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/ppyolov2_r50vd_dcn_mainbody_v1.0_pretrained.pdparams +``` + +* 模型恢复训练 + +在日常训练过程中，有的用户由于一些原因导致训练中断，可以使用 `-r` 的命令恢复训练: + +```bash +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --eval -r output/ppyolov2_r50vd_dcn_365e_coco/10000 +``` + +注意：如果遇到 "`Out of memory error`" 问题, 尝试在 `ppyolov2_reader.yml` 文件中调小 `batch_size`，同时等比例调小学习率。 + + + +### 3.5 模型预测与调试 + +使用下面的命令完成 PaddleDetection 的预测过程。 + +```bash +export CUDA_VISIBLE_DEVICES=0 +python tools/infer.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --infer_img=your_image_path.jpg --output_dir=infer_output/ --draw_threshold=0.5 -o weights=output/ppyolov2_r50vd_dcn_365e_coco/model_final +``` + +`--draw_threshold` 是个可选参数. 根据 [NMS](https://ieeexplore.ieee.org/document/1699659) 的计算，不同阈值会产生不同的结果 `keep_top_k` 表示设置输出目标的最大数量，默认值为 100，用户可以根据自己的实际情况进行设定。 + + + +### 3.6 模型导出与预测部署。 + +执行导出模型脚本： + +```bash +python tools/export_model.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --output_dir=./inference -o weights=output/ppyolov2_r50vd_dcn_365e_coco/model_final.pdparams +``` + +预测模型会导出到 `inference/ppyolov2_r50vd_dcn_365e_coco` 目录下，分别为 `infer_cfg.yml` (预测不需要), `model.pdiparams`, `model.pdiparams.info`, `model.pdmodel` 。 + +注意： `PaddleDetection` 导出的 inference 模型的文件格式为 `model.xxx`，这里如果希望与 PaddleClas 的 inference 模型文件格式保持一致，需要将其 `model.xxx` 文件修改为 `inference.xxx` 文件，用于后续主体检测的预测部署。 + +更多模型导出教程，请参考： [EXPORT_MODEL](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/deploy/EXPORT_MODEL.md) + +最终，目录 `inference/ppyolov2_r50vd_dcn_365e_coco` 中包含 `inference.pdiparams`, `inference.pdiparams.info` 以及 `inference.pdmodel` 文件，其中 `inference.pdiparams` 为保存的 inference 模型权重文件，`inference.pdmodel` 为保存的 inference 模型结构文件。 + + +导出模型之后，在主体检测与识别任务中，就可以将检测模型的路径更改为该 inference 模型路径，完成预测。 + +以商品识别为例，其配置文件为 [inference_product.yaml](../../../deploy/configs/inference_product.yaml)，修改其中的 `Global.det_inference_model_dir` 字段为导出的主体检测 inference 模型目录，参考[图像识别快速开始教程](../quick_start/quick_start_recognition.md)，即可完成商品检测与识别过程。 + + +### FAQ + +#### Q：可以使用其他的主体检测模型结构吗？ + +* A：可以的，但是目前的检测预处理过程仅适配了 PicoDet 以及 YOLO 系列的预处理，因此在使用的时候，建议优先使用这两个系列的模型进行训练，如果希望使用 Faster RCNN 等其他系列的模型，需要按照 PaddleDetection 的数据预处理，修改下预处理逻辑，这块如果您有需求或者有问题的话，欢迎提 issue 或者在微信群里反馈。 + +#### Q：可以修改主体检测的预测尺度吗？ + +* A：可以的，但是需要注意 2 个地方 + * PaddleClas 中提供的主体检测模型是基于 `640x640` 的分辨率去训练的，因此预测的时候也是默认使用 `640x640` 的分辨率进行预测，使用其他分辨率预测的话，精度会有所降低。 + * 在模型导出的时候，建议也修改下模型导出的分辨率，保持模型导出、模型预测的分辨率一致。 diff --git a/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/vector_search.md b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/vector_search.md new file mode 100644 index 0000000..6cf4d20 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/image_recognition_pipeline/vector_search.md @@ -0,0 +1,120 @@ +# 向量检索 + +向量检索技术在图像识别、图像检索中应用比较广泛。其主要目标是，对于给定的查询向量，在已经建立好的向量库中，与库中所有的待查询向量，进行特征向量的相似度或距离计算，得到相似度排序。在图像识别系统中，我们使用 [Faiss](https://github.com/facebookresearch/faiss) 对此部分进行支持，具体信息请详查 [Faiss 官网](https://github.com/facebookresearch/faiss)。`Faiss` 主要有以下优势 + +- 适配性好：支持 Windos、Linux、MacOS 系统 +- 安装方便： 支持 `python` 接口，直接使用 `pip` 安装 +- 算法丰富：支持多种检索算法，满足不同场景的需求 +- 同时支持 CPU、GPU，能够加速检索过程 + +值得注意的是，为了更好是适配性，目前版本，`PaddleClas` 中暂时**只使用 CPU 进行向量检索**。 + +![](../../images/structure.jpg) + +如上图中所示，向量检索部分，在整个 `PP-ShiTu` 系统中有两部分内容 + +- 图中绿色部分：建立检索库，供检索时查询使用，同时提供增、删等功能 +- 图中蓝色部分：检索功能，即给定一张图的特征向量，返回库中相似图像的 label + +本文档主要主要介绍 PaddleClas 中检索模块的安装、使用的检索算法、建库流程的及相关配置文件中参数介绍。 + +-------------------------- + +## 目录 + +- [1. 检索库安装](#1) +- [2. 使用的检索算法](#2) +- [3. 使用及配置文档介绍](#3) + - [3.1 建库及配置文件参数](#3.1) + - [3.2 检索配置文件参数](#3.2) + + + +## 1. 检索库安装 + +`Faiss` 具体安装方法如下： + +```python +pip install faiss-cpu==1.7.1post2 +``` + +若使用时，不能正常引用，则 `uninstall` 之后，重新 `install`，尤其是 `windows` 下。 + + + +## 2. 使用的检索算法 + +目前 `PaddleClas` 中检索模块，支持如下三种检索算法 + +- **HNSW32**: 一种图索引方法。检索精度较高，速度较快。但是特征库只支持添加图像功能，不支持删除图像特征功能。（默认方法） +- **IVF**：倒排索引检索方法。速度较快，但是精度略低。特征库支持增加、删除图像特征功能。 +- **FLAT**： 暴力检索算法。精度最高，但是数据量大时，检索速度较慢。特征库支持增加、删除图像特征功能。 + +每种检索算法，满足不同场景。其中 `HNSW32` 为默认方法，此方法的检索精度、检索速度可以取得一个较好的平衡，具体算法介绍可以查看[官方文档](https://github.com/facebookresearch/faiss/wiki)。 + + + +## 3. 使用及配置文档介绍 + +涉及检索模块配置文件位于：`deploy/configs/` 下，其中 `build_*.yaml` 是建立特征库的相关配置文件，`inference_*.yaml` 是检索或者分类的推理配置文件。 + + + +### 3.1 建库及配置文件参数 + +建库的具体操作如下： + +```shell +# 进入 deploy 目录 +cd deploy +# yaml 文件根据需要改成自己所需的具体 yaml 文件 +python python/build_gallery.py -c configs/build_***.yaml +``` + +其中 `yaml` 文件的建库的配置如下，在运行时，请根据实际情况进行修改。建库操作会将根据 `data_file` 的图像列表，将 `image_root` 下的图像进行特征提取，并在 `index_dir` 下进行存储，以待后续检索使用。 + +其中 `data_file` 文件存储的是图像文件的路径和标签，每一行的格式为：`image_path label`。中间间隔以 `yaml` 文件中 `delimiter` 参数作为间隔。 + +关于特征提取的具体模型参数，可查看 `yaml` 文件。 + +```yaml +# indexing engine config +IndexProcess: + index_method: "HNSW32" # supported: HNSW32, IVF, Flat + index_dir: "./recognition_demo_data_v1.1/gallery_product/index" + image_root: "./recognition_demo_data_v1.1/gallery_product/" + data_file: "./recognition_demo_data_v1.1/gallery_product/data_file.txt" + index_operation: "new" # suported: "append", "remove", "new" + delimiter: "\t" + dist_type: "IP" + embedding_size: 512 +``` + +- **index_method**：使用的检索算法。目前支持三种，HNSW32、IVF、Flat +- **index_dir**：构建的特征库所存放的文件夹 +- **image_root**：构建特征库所需要的标注图像所存储的文件夹位置 +- **data_file**：构建特征库所需要的标注图像的数据列表，每一行的格式：relative_path label +- **index_operation**： 此次运行建库的操作：`new` 新建，`append` 将 data_file 的图像特征添加到特征库中，`remove` 将 data_file 的图像从特征库中删除 +- **delimiter**：**data_file** 中每一行的间隔符 +- **dist_type**: 特征匹配过程中使用的相似度计算方式。例如 `IP` 内积相似度计算方式，`L2` 欧式距离计算方法 +- **embedding_size**：特征维度 + + + +### 3.2 检索配置文件参数 + +将检索的过程融合到 `PP-ShiTu` 的整体流程中，请参考 [README](../../../README_ch.md) 中 `PP-ShiTu 图像识别系统介绍` 部分。检索具体使用操作请参考[识别快速开始文档](../quick_start/quick_start_recognition.md)。 + +其中，检索部分配置如下，整体检索配置文件，请参考 `deploy/configs/inference_*.yaml` 文件。 + +```yaml +IndexProcess: + index_dir: "./recognition_demo_data_v1.1/gallery_logo/index/" + return_k: 5 + score_thres: 0.5 +``` + +与建库配置文件不同，新参数主要如下： + +- `return_k`: 检索结果返回 `k` 个结果 +- `score_thres`: 检索匹配的阈值 diff --git a/src/PaddleClas/docs/zh_CN/index.rst b/src/PaddleClas/docs/zh_CN/index.rst new file mode 100644 index 0000000..a41ca9b --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/index.rst @@ -0,0 +1,18 @@ +欢迎使用PaddleClas图像分类库！ +================================ + +.. toctree:: + :maxdepth: 1 + + introduction/index + installation/index + quick_start/index + image_recognition_pipeline/index + data_preparation/index + models_training/index + inference_deployment/index + models/index + algorithm_introduction/index + advanced_tutorials/index + others/index + faq_series/index diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/cpp_deploy.md b/src/PaddleClas/docs/zh_CN/inference_deployment/cpp_deploy.md new file mode 100644 index 0000000..cb7e9f3 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/cpp_deploy.md @@ -0,0 +1,298 @@ +# 服务器端 C++ 预测 + +本教程将介绍在服务器端部署 PaddleClas 分类模型的详细步骤，识别模型部署方式将在近期支持，敬请期待。 + +--- + +## 目录 + +- [1. 准备环境](#1) + - [1.1 编译 opencv 库](#1.1) + - [1.2 准备环境](#1.2) + - [1.2.1 预测库源码编译](#1.2.1) + - [1.2.2 直接下载安装](#1.2.2) +- [2. 编译](#2) + - [2.1 编译 PaddleClas C++ 预测 demo](#2.1) + - [2.2 编译 config lib 预测库与 cls lib 预测库](#2.2) +- [3. 运行](#3) + - [3.1 准备 inference model](#3.1) + - [3.2 运行 demo](#3.2) + + + +## 1. 准备环境 + +- Linux 环境，推荐使用 docker。 +- Windows 环境，目前支持基于 `Visual Studio 2019 Community` 进行编译；此外，如果您希望通过生成 `sln 解决方案` 的方式进行编译，可以参考该文档：[https://zhuanlan.zhihu.com/p/145446681](https://zhuanlan.zhihu.com/p/145446681) + +* 该文档主要介绍基于 Linux 环境下的 PaddleClas C++ 预测流程，如果需要在 Windows 环境下使用预测库进行 C++ 预测，具体编译方法请参考 [Windows 下编译教程](./cpp_deploy_on_windows.md)。 + + +### 1.1 编译 opencv 库 + +* 首先需要从 opencv 官网上下载在 Linux 环境下源码编译的包，以 3.4.7 版本为例，下载及解压缩命令如下： + +``` +wget https://github.com/opencv/opencv/archive/3.4.7.tar.gz +tar -xvf 3.4.7.tar.gz +``` + +执行完成上述命令后，会在当前目录下生成 `opencv-3.4.7/` 的目录。 + +* 编译 opencv，首先设置 opencv 源码路径(`root_path`)以及安装路径(`install_path`)，`root_path` 为下载的 opencv 源码路径，`install_path` 为 opencv 的安装路径。在本例中，源码路径即为当前目录下的 `opencv-3.4.7/`。 + +```shell +cd ./opencv-3.4.7 +export root_path=$PWD +export install_path=${root_path}/opencv3 +``` + +* 然后在 opencv 源码路径下，按照下面的命令进行编译： + +```shell +rm -rf build +mkdir build +cd build + +cmake .. \ + -DCMAKE_INSTALL_PREFIX=${install_path} \ + -DCMAKE_BUILD_TYPE=Release \ + -DBUILD_SHARED_LIBS=OFF \ + -DWITH_IPP=OFF \ + -DBUILD_IPP_IW=OFF \ + -DWITH_LAPACK=OFF \ + -DWITH_EIGEN=OFF \ + -DCMAKE_INSTALL_LIBDIR=lib64 \ + -DWITH_ZLIB=ON \ + -DBUILD_ZLIB=ON \ + -DWITH_JPEG=ON \ + -DBUILD_JPEG=ON \ + -DWITH_PNG=ON \ + -DBUILD_PNG=ON \ + -DWITH_TIFF=ON \ + -DBUILD_TIFF=ON + +make -j +make install +``` + +* 执行完成上述命令后，会生成 opencv 头文件和库文件，用于后面的 PaddleClas 代码编译。 + +以 opencv3.4.7 版本为例，最终在安装路径下的文件结构如下所示。**注意**：不同的 opencv 版本，下述的文件结构可能不同。 + +``` +opencv3/ +|-- bin +|-- include +|-- lib64 +|-- share +``` + + +### 1.2 获取 Paddle 预测库 + +* 有 2 种方式获取 Paddle 预测库，下面进行详细介绍。 + + +#### 1.2.1 预测库源码编译 +如果希望获取最新预测库特性，可以从 GitHub 上克隆 Paddle 最新代码，从源码编译预测库。对于不同平台的编译流程，请参考 [Paddle 预测库官网](https://paddleinference.paddlepaddle.org.cn/v2.1/user_guides/source_compile.html) 的说明。编译示例如下： + +1. 使用 Git 获取源代码： + +```shell +git clone https://github.com/PaddlePaddle/Paddle.git +cd Paddle +``` + +2. 进入 `Paddle` 目录后，开始进行编译： + +```shell +rm -rf build +mkdir build +cd build + +cmake .. \ + -DWITH_CONTRIB=OFF \ + -DWITH_MKL=ON \ + -DWITH_MKLDNN=ON \ + -DWITH_TESTING=OFF \ + -DCMAKE_BUILD_TYPE=Release \ + -DWITH_INFERENCE_API_TEST=OFF \ + -DON_INFER=ON \ + -DWITH_PYTHON=ON +make -j +make inference_lib_dist +``` + +关于编译参数选项可以参考 Paddle C++ 预测库官网：[编译选项表](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#bianyixuanxiangbiao)。 + + +3. 编译完成之后，可以在 `./build/paddle_inference_install_dir/` 文件下看到以下文件及目录： + +``` +build/paddle_inference_install_dir/ +|-- CMakeCache.txt +|-- paddle +|-- third_party +|-- version.txt +``` + + +#### 1.2.2 直接下载安装 + +[Paddle 预测库官网](https://paddleinference.paddlepaddle.org.cn/v2.1/user_guides/download_lib.html#c) 上提供了不同版本的 Paddle 预测库，包括多个操作系统平台和 GPU、CPU 等多个硬件平台的预编译库，可以在官网查找并选择合适的预测库版本进行下载，建议选择 `2.1.1` 版本。 + +**注意**：在选择预测库时，所选预测库版本需要与后续编译选项一致： +* CPU 预测库仅可用于 GPU 预测，具体又分为 `mkl` 和 `openblas`，分别对应其低层实现基于 `MKL` 数学运算库 和 `OpenBLAS` 数学运算库； +* GPU 预测库支持 GPU 预测和 CPU 预测，其 GPU 预测功能底层实现基于 CUDA、cuDNN，CPU 预测功能底层基于 `MKL` 实现。 + +以 `manylinux_cuda10.2_cudnn8.1_avx_mkl_trt7_gcc8.2` 的 `2.1.1` 版本为例，可使用下述命令下载并解压： + +```shell +wget https://paddle-inference-lib.bj.bcebos.com/2.1.1-gpu-cuda10.2-cudnn8.1-mkl-gcc8.2/paddle_inference.tgz +tar -xf paddle_inference.tgz +``` + +上述命令会在当前目录下生成 `paddle_inference/` 目录。 + + +## 2. 编译 + + +### 2.1 编译 PaddleClas C++ 预测 demo + +编译命令如下，其中 Paddle C++ 预测库、OpenCV 等依赖库的路径需要根据机器上的实际位置进行修改。 + +```shell +sh tools/build_demo.sh +``` + +具体地，`tools/build_demo.sh` 中内容如下： + +```shell +OpenCV_DIR=path/to/opencv +PADDLE_LIB_DIR=path/to/paddle +CUDA_LIB_DIR=path/to/cuda +CUDNN_LIB_DIR=path/to/cudnn +TENSORRT_LIB_DIR=path/to/tensorrt +CONFIG_LIB_PATH=path/to/config/library +CLS_LIB_PATH=path/to/cls/library +CMP_STATIC=ON + +BUILD_DIR=build +rm -rf ${BUILD_DIR} +mkdir ${BUILD_DIR} +cd ${BUILD_DIR} +cmake .. \ + -DWITH_MKL=ON \ + -DWITH_GPU=ON \ + -DWITH_TENSORRT=OFF \ + -DWITH_STATIC_LIB=OFF \ + -DOpenCV_DIR=${OpenCV_DIR} \ + -DPADDLE_LIB=${PADDLE_LIB_DIR} \ + -DCUDA_LIB=${CUDA_LIB_DIR} \ + -DCUDNN_LIB=${CUDNN_LIB_DIR} \ + -DCONFIG_LIB=${CONFIG_LIB_PATH} \ + -DCLS_LIB=${CLS_LIB_PATH} \ + -DCMP_STATIC=OFF \ + +make -j +``` + +上述命令中，各个编译选项的具体说明如下： +* `DWITH_MKL`：是否使用 MKL 数学运算库，默认为 `ON`，注意需要与下载的预测库版本一致。如为 `OpenBLAS` 版本预测库，则需设置为 `OFF`； +* `DWITH_GPU`：使用使用 GPU 预测库，默认为 `OFF`，当下载预测库版本为 GPU 预测库时，可置为 `ON`，同时请设置 `DCUDA_LIB` 与 `DCUDNN_LIB`，否则请设置为 `OFF`； +* `DWITH_TENSORRT`：是否使用 TensorRT，默认为 `OFF`，当 `DWITH_GPU` 设置为 `ON` 时有效，同时请设置 `DTENSORRT_LIB`； +* `DWITH_STATIC_LIB`：是否使用 Paddle 预测库中的静态链接库，默认为 `OFF`； +* `DOpenCV_DIR`：OpenCV 编译库的路径，本例中为 `opencv-3.4.7/opencv3/share/OpenCV`，注意该路径下需要有 `OpenCVConfig.cmake` 文件； +* `DPADDLE_LIB`：Paddle 预测库路径，一般使用下载并解压的预编译库路径 `paddle_inference` 即可，或编译生成的预测库的路径 `build/paddle_inference_install_dir`，注意该路径下需要有 `paddle` 和 `third_party` 两个子路径； +* `DCUDA_LIB`：CUDA 库路径，当 `DWITH_GPU` 设置为 `ON` 时有效，在 docker 中通常为 `/usr/local/cuda/lib64`； +* `DCUDNN_LIB`：cuDNN 库路径，当 `DWITH_GPU` 设置为 `ON` 时有效，在 docker 中通常为 `/usr/lib/x86_64-linux-gnu/`； +* `DTENSORRT_LIB`：TensorRT 库路径，当 `DWITH_GPU` 与 `DUSE_TENSORRT` 均设置为 `ON` 时有效，在 dokcer 中通常为 `/usr/local/TensorRT6-cuda10.0-cudnn7/`； +* `DCONFIG_LIB`：如需使用已编译好的 `config lib`，请设置为 `config lib` 的路径，否则请删除该选项； +* `DCLS_LIB`：如需使用已编译好的 `cls lib`，请设置为 `cls lib` 的路径，否则请删除该选项； +* `DCMP_STATIC`：是否将 `config lib` 和 `cls lib` 编译为静态链接库，默认为 `ON`，如需编译为动态链接库，请设置为 `OFF`。 + +在执行上述命令，编译完成之后，会在当前路径下生成 `build` 目录，其中有可执行文件 `clas_system`。并且，如未设置 `DCONFIG_LIB` 与 `DCLS_LIB`，则会在 `.\lib\` 路径下生成 `config lib` 和 `cls lib` 两个库文件，根据 `DCMP_STATIC` 的设置，两个库文件将被编译为静态链接库（`libconfig.a`、`libcls.a`）或动态链接库（`libconfig.so`、`libcls.so`）。 + + +### 2.2 编译 config lib 预测库与 cls lib 预测库 + +除了可以直接编译 demo，也同样可以仅编译 config lib 和 cls lib 预测库，只需运行如下命令： + +```shell +sh tools/build_lib.sh +``` + +具体地，`tools/build_lib.sh` 中内容如下： + +```shell +OpenCV_DIR=path/to/opencv +PADDLE_LIB_DIR=path/to/paddle + +BUILD_DIR=./lib/build +rm -rf ${BUILD_DIR} +mkdir ${BUILD_DIR} +cd ${BUILD_DIR} +cmake .. \ + -DOpenCV_DIR=${OpenCV_DIR} \ + -DPADDLE_LIB=${PADDLE_LIB_DIR} \ + -DCMP_STATIC=ON \ + +make +``` + +上述命令中，各个编译选项的具体说明如下： +* `DOpenCV_DIR`：OpenCV 编译库的路径，本例中为 `opencv-3.4.7/opencv3/share/OpenCV`，注意该路径下需要有 `OpenCVConfig.cmake` 文件； +* `DPADDLE_LIB`：Paddle 预测库路径，一般使用下载并解压的预编译库路径 `paddle_inference` 即可，或编译生成的预测库的路径 `build/paddle_inference_install_dir`，注意该路径下需要有 `paddle` 和 `third_party` 两个子路径； +* `DCMP_STATIC`：是否将 `config lib` 和 `cls lib` 编译为静态链接库（`.a`），默认为 `ON`，如需编译为动态链接库（`.so`），请设置为 `OFF`。 + +执行上述命令后，将在目录 `./lib/` 下生成 `config lib` 和 `cls lib` 的动态链接库（`libcls.so` 和 `libconfig.so`）或静态链接库（`libcls.a` 和 `libconfig.a`）。在[2.1 编译 PaddleClas C++ 预测 demo](#2.1)中，可以通过指定编译选项 `DCLS_LIB` 和 `DCONFIG_LIB` 指定已有链接库的路径，链接库同样也可用于二次开发。 + + +## 3. 运行 + + +### 3.1 准备 inference model + +首先需要准备 inference model，关于将模型导出为 inference model 的具体步骤，可以参考 [模型导出](./export_model.md) 文档。假设导出的预测模型文件放在 `./inference` 目录下，则目录结构如下。 + +``` +inference/ +|--inference.pdmodel +|--inference.pdiparams +``` + +**注意**：上述文件中，`cls_infer.pdmodel` 文件存储了模型网络结构信息，`cls_infer.pdiparams` 文件存储了模型参数权重信息。在运行预测时，注意两个文件的路径需要分别设置为配置文件 `tools/config.txt` 中的字段 `cls_model_path` 和 `cls_params_path` 的值。 + + +### 3.2 运行 demo + +首先修改 `tools/config.txt` 中对应字段： +* use_gpu：是否使用 GPU； +* gpu_id：使用的 GPU 卡号； +* gpu_mem：显存； +* cpu_math_library_num_threads：底层科学计算库所用线程的数量； +* use_mkldnn：是否使用 MKLDNN 加速； +* use_tensorrt: 是否使用 tensorRT 进行加速； +* use_fp16：是否使用半精度浮点数进行计算，该选项仅在 use_tensorrt 为 true 时有效； +* cls_model_path：预测模型结构文件路径； +* cls_params_path：预测模型参数文件路径； +* resize_short_size：预处理时图像缩放大小； +* crop_size：预处理时图像裁剪后的大小。 + +然后修改 `tools/run.sh`： + * `./build/clas_system ./tools/config.txt ./docs/imgs/ILSVRC2012_val_00000666.JPEG` + * 上述命令中分别为：编译得到的可执行文件 `clas_system`；运行时的配置文件 `config.txt`；待预测的图像。 + +* 最后执行以下命令，完成对一幅图像的分类。 + +```shell +sh tools/run.sh +``` + +* 最终屏幕上会输出结果，如下图所示。 + +![](../../images/inference_deployment/cpp_infer_result.png) + +其中 `class id` 表示置信度最高的类别对应的 id，score 表示图片属于该类别的概率。 diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/cpp_deploy_on_windows.md b/src/PaddleClas/docs/zh_CN/inference_deployment/cpp_deploy_on_windows.md new file mode 100644 index 0000000..b7089cb --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/cpp_deploy_on_windows.md @@ -0,0 +1,165 @@ +# 基于 Visual Studio 2019 Community CMake 编译指南 + +PaddleClas 在 Windows 平台下基于 `Visual Studio 2019 Community` 进行了测试。微软从 `Visual Studio 2017` 开始即支持直接管理 `CMake` 跨平台编译项目，但是直到 `2019` 版才提供了稳定和完全的支持，所以如果你想使用 CMake 管理项目编译构建，我们推荐使用 `Visual Studio 2019`。如果您希望通过生成 `sln 解决方案` 的方式进行编译，可以参考该文档：[https://zhuanlan.zhihu.com/p/145446681](https://zhuanlan.zhihu.com/p/145446681)。 + +----- +## 目录 +* [1. 前置条件](#1) + * [1.1 下载 PaddlePaddle C++ 预测库 paddle_inference_install_dir](#1.1) + * [1.2 安装配置 OpenCV](#1.2) +* [2. 使用 Visual Studio 2019 编译](#2) +* [3. 预测](#3) + * [3.1 准备 inference model](#3.1) + * [3.2 运行预测](#3.2) + * [3.3 注意事项](#3.3) + + +## 1. 前置条件 +* Visual Studio 2019 +* CUDA 9.0 / CUDA 10.0，cudnn 7.6+（仅在使用 GPU 版本的预测库时需要） +* CMake 3.0+ + +请确保系统已经正确安装并配置好上述基本软件，其中： + * 在安装 `Visual Studio 2019` 时，`工作负载` 需要勾选 `使用 C++的桌面开发`； + * CUDA 需要正确安装并设置系统环境变量； + * CMake 需要正确安装并将路径添加到系统环境变量中。 + +以下示例基于 `Visual Studio 2019 Community` 版本，以工作目录为 `D:\projects` 进行演示。 + + +### 1.1 下载 PaddlePaddle C++ 预测库 paddle_inference_install_dir + +PaddlePaddle C++ 预测库针对不同的 `CPU `和 `CUDA` 版本提供了不同的预编译版本，请根据实际情况下载: [C++预测库下载列表](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html#windows)，建议选择 `2.1.1` 版本。 + +**注意**：在选择预测库时，所选预测库版本需要与后续编译选项一致： +* CPU 预测库仅可用于 GPU 预测，具体又分为 `mkl` 和 `openblas`，分别对应其低层实现基于 `MKL` 数学运算库 和 `OpenBLAS` 数学运算库； +* GPU 预测库支持 GPU 预测和 CPU 预测，其 GPU 预测功能底层实现基于 CUDA、cuDNN，CPU 预测功能底层基于 `MKL` 实现。 + +下载并解压后，目录 `D:\projects\paddle_inference_install_dir` 包含内容为： + +``` +paddle_inference_install_dir +├── paddle # paddle 核心库和头文件 +├── third_party # 第三方依赖库和头文件 +└── version.txt # 版本和编译信息 +``` + +**注意**：需要将 `Paddle 预测库` 的路径（`D:\projects\paddle_inference_install_dir\paddle\lib`）添加到系统环境变量 `Path` 中。 + + +### 1.2 安装配置 OpenCV + +1. 在 OpenCV 官网下载适用于 Windows 平台的 3.4.6 版本，[下载地址](https://sourceforge.net/projects/opencvlibrary/files/3.4.6/opencv-3.4.6-vc14_vc15.exe/download)； +2. 运行下载的可执行文件，将 OpenCV 解压至指定目录，如 `D:\projects\opencv`； +3. 配置环境变量，如下流程所示： + * 此电脑（我的电脑）-> 属性 -> 高级系统设置 -> 环境变量； + * 在系统变量中找到 Path（如没有，自行创建），并双击编辑； + * 新建，将 OpenCV 路径填入并保存，如 `D:\projects\opencv\build\x64\vc14\bin`； + + +## 2. 使用 Visual Studio 2019 编译 + +1. 打开 Visual Studio 2019 Community，点击 `继续但无需代码` + +![step2](../../images/inference_deployment/vs2019_step1.png) + +2. 点击：`文件`->`打开`->`CMake` + +![step2.1](../../images/inference_deployment/vs2019_step2.png) + +选择项目代码所在路径，并打开 `CMakeList.txt`： + +![step2.2](../../images/inference_deployment/vs2019_step3.png) + +3. 点击：`项目`->`CMake 设置` + +![step3](../../images/inference_deployment/vs2019_step4.png) + +4. 请设置以下参数的值 + +| 名称 | 值 | 保存到 JSON | +| ----------------------------- | ------------------ | ----------- | +| CMAKE_BACKWARDS_COMPATIBILITY | 3.17 | [√] | +| CMAKE_BUILD_TYPE | RelWithDebInfo | [√] | +| CUDA_LIB | CUDA 的库路径 | [√] | +| CUDNN_LIB | CUDNN 的库路径 | [√] | +| OpenCV_DIR | OpenCV 的安装路径 | [√] | +| PADDLE_LIB | Paddle 预测库的路径 | [√] | +| WITH_GPU | [√] | [√] | +| WITH_MKL | [√] | [√] | +| WITH_STATIC_LIB | [√] | [√] | + +除上述选项外，还有以下两个选项可依据具体情况设置： +* `DCONFIG_LIB`：如需使用已编译好的 `config lib`，请设置为 `config lib` 的路径，否则请删除该选项； +* `DCLS_LIB`：如需使用已编译好的 `cls lib`，请设置为 `cls lib` 的路径，否则请删除该选项； + +**注意**： +* `CMAKE_BACKWARDS_COMPATIBILITY` 的值请根据自己 `cmake` 版本设置，`cmake` 版本可以通过命令：`cmake --version` 查询； +* `CUDA_LIB` 、 `CUDNN_LIB` 的值仅需在使用 **GPU 版本**预测库时指定，其中 CUDA 库版本尽量对齐，**使用 9.0、10.0 版本，不使用 9.2、10.1 等版本 CUDA 库**； +* 在设置 `CUDA_LIB`、`CUDNN_LIB`、`OPENCV_DIR`、`PADDLE_LIB` 时，点击 `浏览`，分别设置相应的路径； + * `CUDA_LIB` 和 `CUDNN_LIB`：该路径取决于 CUDA 与 CUDNN 的安装位置。 + * `OpenCV_DIR`：该路径下需要有`.cmake` 文件，一般为 `opencv/build/`； + * `PADDLE_LIB`：该路径下需要有 `CMakeCache.txt` 文件，一般为 `paddle_inference_install_dir/`。 +* 在使用 `CPU` 版预测库时，请不要勾选 `WITH_GPU` - `保存到 JSON`。 + +![step4](../../images/inference_deployment/vs2019_step5.png) + +设置完成后，点击上图中 `保存并生成 CMake 缓存以加载变量`。 + +5. 点击`生成`->`全部生成` + +![step6](../../images/inference_deployment/vs2019_step6.png) + +在编译完成后，会生成可执行文件 `clas_system.exe`。并且，如未设置 `DCONFIG_LIB` 与 `DCLS_LIB`，则会在 `.\lib\` 目录下生成 `config lib` 和 `cls lib` 两个静态链接库文件（`libconfig.a`、`libcls.a`）。类似地，你也可以仅编译生成 `config lib` 和 `cls lib` 两个静态链接库文件，只需打开路径为 `D:\projects\PaddleClas\deploy\cpp\lib\CMakeList.txt` 的 `CMake` 文件并进行编译即可，具体参考[2. 使用 Visual Studio 2019 编译](#2)，完成编译后，同样可在 `.\lib\` 目录下生成静态链接库文件，静态链接库文件可用于二次开发。 + + +## 3. 预测 + + +### 3.1 准备 inference model + +首先需要准备 inference model，关于将模型导出为 inference model 的具体步骤，可以参考 [模型导出](./export_model.md) 文档。假设导出的预测模型文件放在 `./inference` 目录下，则目录结构如下。 + +``` +inference/ +|--inference.pdmodel +|--inference.pdiparams +``` + +**注意**：上述文件中，`cls_infer.pdmodel` 文件存储了模型网络结构信息，`cls_infer.pdiparams` 文件存储了模型参数权重信息。在运行预测时，注意两个文件的路径需要分别设置为配置文件 `tools/config.txt` 中的字段 `cls_model_path` 和 `cls_params_path` 的值。 + + +### 3.2 运行预测 + +首先修改 `tools/config.txt` 中对应字段： +* use_gpu：是否使用 GPU； +* gpu_id：使用的 GPU 卡号； +* gpu_mem：显存； +* cpu_math_library_num_threads：底层科学计算库所用线程的数量； +* use_mkldnn：是否使用 MKLDNN 加速； +* use_tensorrt: 是否使用 tensorRT 进行加速； +* use_fp16：是否使用半精度浮点数进行计算，该选项仅在 use_tensorrt 为 true 时有效； +* cls_model_path：预测模型结构文件路径； +* cls_params_path：预测模型参数文件路径； +* resize_short_size：预处理时图像缩放大小； +* crop_size：预处理时图像裁剪后的大小。 + +`Visual Studio 2019` 编译产出的可执行文件 `clas_system.exe` 在 `out\build\x64-Release` 目录下，打开 `cmd`，并切换到该目录： + +```shell +cd D:\projects\PaddleClas\deploy\cpp\out\build\x64-Release +``` + +可执行文件 `clas_system.exe` 即为编译产出的的预测程序，运行下述命令即可执行预测： +```shell +.\clas_system.exe D:\projects\PaddleClas\deploy\cpp\tools\config.txt .\docs\ILSVRC2012_val_00008306.JPEG +``` + +上述命令中，第一个参数（`D:\projects\PaddleClas\deploy\cpp\tools\config.txt`）为配置文件路径，第二个参数（`.\docs\ILSVRC2012_val_00008306.JPEG`）为需要预测的图片路径。 + +注意，需要在配置文件中正确设置预测参数，包括所用模型文件的路径（`cls_model_path` 和 `cls_params_path`）。 + +### 3.3 注意事项 +* 在 Windows 下的终端中执行文件 exe 时，可能会发生乱码的现象，此时需要在终端中输入 `CHCP 65001`，将终端的编码方式由 GBK 编码(默认)改为 UTF-8 编码，更加具体的解释可以参考这篇博客：[https://blog.csdn.net/qq_35038153/article/details/78430359](https://blog.csdn.net/qq_35038153/article/details/78430359)； +* 如果需要使用 CPU 预测，PaddlePaddle 在 Windows 上仅支持 avx 的 CPU 预测，目前不支持 noavx 的 CPU 预测； +* 在使用生成的 `clas_system.exe` 进行预测时，如提示 `由于找不到 paddle_fluid.dll，无法继续执行代码。重新安装程序可能会解决此问题`，请检查是否将 Paddle 预测库路径添加到系统环境变量，详见[1.1 下载 PaddlePaddle C++ 预测库 paddle_inference_install_dir](#1.1)。 diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/export_model.md b/src/PaddleClas/docs/zh_CN/inference_deployment/export_model.md new file mode 100644 index 0000000..1d8decb --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/export_model.md @@ -0,0 +1,99 @@ +# 模型导出 + +PaddlePaddle 支持导出 inference 模型用于部署推理场景，相比于训练调优场景，inference 模型会将网络权重与网络结构进行持久化存储，并且 PaddlePaddle 支持使用预测引擎加载 inference 模型进行预测推理。 + +--- + + +## 目录 + +- [1. 环境准备](#1) +- [2. 分类模型导出](#2) +- [3. 主体检测模型导出](#3) +- [4. 识别模型导出](#4) +- [5. 命令参数说明](#5) + + + +## 1. 环境准备 + +首先请参考文档[安装 PaddlePaddle](../installation/install_paddle.md)和文档[安装 PaddleClas](../installation/install_paddleclas.md)配置运行环境。 + + +## 2. 分类模型导出 + +进入 PaddleClas 目录下： + +```shell +cd /path/to/PaddleClas +``` + +以 ResNet50_vd 分类模型为例，下载预训练模型： + +```shell +wget -P ./cls_pretrain/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_pretrained.pdparams +``` + +上述模型是使用 ResNet50_vd 在 ImageNet 上训练的模型，使用的配置文件为 `ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml`，将该模型转为 inference 模型只需运行如下命令： + +```shell +python tools/export_model.py \ + -c ./ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml \ + -o Global.pretrained_model=./cls_pretrain/ResNet50_vd_pretrained \ + -o Global.save_inference_dir=./deploy/models/class_ResNet50_vd_ImageNet_infer +``` + + +## 3. 主体检测模型导出 + +主体检测模型的导出，可以参考[主题检测介绍](../image_recognition_pipeline/mainbody_detection.md)。 + + +## 4. 识别模型导出 + +进入 PaddleClas 目录下： + +```shell +cd /path/to/PaddleClas +``` + +以商品识别特征提取模型为例，下载预训练模型： + +```shell +wget -P ./product_pretrain/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/product_ResNet50_vd_Aliproduct_v1.0_pretrained.pdparams +``` + +上述模型是 ResNet50_vd 在 AliProduct 上训练的模型，训练使用的配置文件为 `ppcls/configs/Products/ResNet50_vd_Aliproduct.yaml`，将该模型转为 inference 模型只需运行如下命令： + +```shell +python3 tools/export_model.py \ + -c ./ppcls/configs/Products/ResNet50_vd_Aliproduct.yaml \ + -o Global.pretrained_model=./product_pretrain/product_ResNet50_vd_Aliproduct_v1.0_pretrained \ + -o Global.save_inference_dir=./deploy/models/product_ResNet50_vd_aliproduct_v1.0_infer +``` + +注意，此处保存的 inference 模型在 embedding 特征层做了截断，即导出后模型最终的输出为 n 维 embedding 特征。 + + +## 5. 命令参数说明 + +在上述模型导出命令中，所使用的配置文件需要与该模型的训练文件相同，在配置文件中有以下字段用于配置模型导出参数： + +* `Global.image_shape`：用于指定模型的输入数据尺寸，该尺寸不包含 batch 维度； +* `Global.save_inference_dir`：用于指定导出的 inference 模型的保存位置； +* `Global.pretrained_model`：用于指定训练过程中保存的模型权重文件路径，该路径无需包含模型权重文件后缀名 `.pdparams`。。 + +上述命令将生成以下三个文件： + +* `inference.pdmodel`：用于存储网络结构信息； +* `inference.pdiparams`：用于存储网络权重信息； +* `inference.pdiparams.info`：用于存储模型的参数信息，在分类模型和识别模型中可忽略。 + +导出的 inference 模型文件可用于预测引擎进行推理部署，根据不同的部署方式/平台，可参考： + +* [Python 预测](./python_deploy.md) +* [C++ 预测](./cpp_deploy.md)（目前仅支持分类模型） +* [Python Whl 预测](./whl_deploy.md)（目前仅支持分类模型） +* [PaddleHub Serving 部署](./paddle_hub_serving_deploy.md)（目前仅支持分类模型） +* [PaddleServing 部署](./paddle_serving_deploy.md) +* [PaddleLite 部署](./paddle_lite_deploy.md)（目前仅支持分类模型） diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/index.rst b/src/PaddleClas/docs/zh_CN/inference_deployment/index.rst new file mode 100644 index 0000000..beddc13 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/index.rst @@ -0,0 +1,17 @@ +推理部署 +================================ + +.. toctree:: + :maxdepth: 2 + + export_model.md + python_deploy.md + cpp_deploy.md + paddle_serving_deploy.md + paddle_hub_serving_deploy.md + paddle_lite_deploy.md + whl_deploy.md + cpp_deploy_on_windows.md + + + diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md b/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md new file mode 100644 index 0000000..1589bc6 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md @@ -0,0 +1,232 @@ +# 基于 PaddleHub Serving 的服务部署 + +PaddleClas 支持通过 PaddleHub 快速进行服务化部署。目前支持图像分类的部署，图像识别的部署敬请期待。 + +--- + + +## 目录 +- [1. 简介](#1) +- [2. 准备环境](#2) +- [3. 下载推理模型](#3) +- [4. 安装服务模块](#4) +- [5. 启动服务](#5) + - [5.1 命令行命令启动](#5.1) + - [5.2 配置文件启动](#5.2) +- [6. 发送预测请求](#6) +- [7. 自定义修改服务模块](#7) + + + +## 1. 简介 + +hubserving 服务部署配置服务包 `clas` 下包含 3 个必选文件，目录如下： + +``` +hubserving/clas/ + └─ __init__.py 空文件，必选 + └─ config.json 配置文件，可选，使用配置启动服务时作为参数传入 + └─ module.py 主模块，必选，包含服务的完整逻辑 + └─ params.py 参数文件，必选，包含模型路径、前后处理参数等参数 +``` + + + +## 2. 准备环境 +```shell +# 安装 paddlehub,请安装 2.0 版本 +pip3 install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple +``` + + + +## 3. 下载推理模型 + +安装服务模块前，需要准备推理模型并放到正确路径，默认模型路径为： + +* 分类推理模型结构文件：`PaddleClas/inference/inference.pdmodel` +* 分类推理模型权重文件：`PaddleClas/inference/inference.pdiparams` + +**注意**： +* 模型文件路径可在 `PaddleClas/deploy/hubserving/clas/params.py` 中查看和修改： + + ```python + "inference_model_dir": "../inference/" + ``` +需要注意， + * 模型文件（包括 `.pdmodel` 与 `.pdiparams`）名称必须为 `inference`。 + * 我们也提供了大量基于 ImageNet-1k 数据集的预训练模型，模型列表及下载地址详见[模型库概览](../models/models_intro.md)，也可以使用自己训练转换好的模型。 + + + +## 4. 安装服务模块 + +针对 Linux 环境和 Windows 环境，安装命令如下。 + +* 在 Linux 环境下，安装示例如下： +```shell +cd PaddleClas/deploy +# 安装服务模块： +hub install hubserving/clas/ +``` + +* 在 Windows 环境下(文件夹的分隔符为`\`)，安装示例如下： + +```shell +cd PaddleClas\deploy +# 安装服务模块： +hub install hubserving\clas\ +``` + + + +## 5. 启动服务 + + + +### 5.1 命令行命令启动 + +该方式仅支持使用 CPU 预测。启动命令： + +```shell +$ hub serving start --modules Module1==Version1 \ + --port XXXX \ + --use_multiprocess \ + --workers \ +``` + +**参数说明**： +|参数|用途| +|-|-| +|--modules/-m| [**必选**] PaddleHub Serving 预安装模型，以多个 Module==Version 键值对的形式列出
*`当不指定 Version 时，默认选择最新版本`*| +|--port/-p| [**可选**] 服务端口，默认为 8866| +|--use_multiprocess| [**可选**] 是否启用并发方式，默认为单进程方式，推荐多核 CPU 机器使用此方式
*`Windows 操作系统只支持单进程方式`*| +|--workers| [**可选**] 在并发方式下指定的并发任务数，默认为 `2*cpu_count-1`，其中 `cpu_count` 为 CPU 核数| + +如按默认参数启动服务：```hub serving start -m clas_system``` + +这样就完成了一个服务化 API 的部署，使用默认端口号 8866。 + + + +### 5.2 配置文件启动 + +该方式仅支持使用 CPU 或 GPU 预测。启动命令： + +```hub serving start -c config.json``` + +其中，`config.json` 格式如下： + +```json +{ + "modules_info": { + "clas_system": { + "init_args": { + "version": "1.0.0", + "use_gpu": true, + "enable_mkldnn": false + }, + "predict_args": { + } + } + }, + "port": 8866, + "use_multiprocess": false, + "workers": 2 +} +``` + +**参数说明**： +* `init_args` 中的可配参数与 `module.py` 中的 `_initialize` 函数接口一致。其中， + - 当 `use_gpu` 为 `true` 时，表示使用 GPU 启动服务。 + - 当 `enable_mkldnn` 为 `true` 时，表示使用 MKL-DNN 加速。 +* `predict_args` 中的可配参数与 `module.py` 中的 `predict` 函数接口一致。 + +**注意**： +* 使用配置文件启动服务时，将使用配置文件中的参数设置，其他命令行参数将被忽略； +* 如果使用 GPU 预测(即，`use_gpu` 置为 `true`)，则需要在启动服务之前，设置 `CUDA_VISIBLE_DEVICES` 环境变量来指定所使用的 GPU 卡号，如：`export CUDA_VISIBLE_DEVICES=0`； +* **`use_gpu` 不可与 `use_multiprocess` 同时为 `true`**； +* **`use_gpu` 与 `enable_mkldnn` 同时为 `true` 时，将忽略 `enable_mkldnn`，而使用 GPU**。 + +如使用 GPU 3 号卡启动服务： + +```shell +cd PaddleClas/deploy +export CUDA_VISIBLE_DEVICES=3 +hub serving start -c hubserving/clas/config.json +``` + + +## 6. 发送预测请求 + +配置好服务端后，可使用以下命令发送预测请求，获取预测结果： + +```shell +cd PaddleClas/deploy +python hubserving/test_hubserving.py server_url image_path +``` + +**脚本参数说明**： +* **server_url**：服务地址，格式为 +`http://[ip_address]:[port]/predict/[module_name]` +* **image_path**：测试图像路径，可以是单张图片路径，也可以是图像集合目录路径。 +* **batch_size**：[**可选**] 以 `batch_size` 大小为单位进行预测，默认为 `1`。 +* **resize_short**：[**可选**] 预处理时，按短边调整大小，默认为 `256`。 +* **crop_size**：[**可选**] 预处理时，居中裁剪的大小，默认为 `224`。 +* **normalize**：[**可选**] 预处理时，是否进行 `normalize`，默认为 `True`。 +* **to_chw**：[**可选**] 预处理时，是否调整为 `CHW` 顺序，默认为 `True`。 + +**注意**：如果使用 `Transformer` 系列模型，如 `DeiT_***_384`, `ViT_***_384` 等，请注意模型的输入数据尺寸，需要指定`--resize_short=384 --crop_size=384`。 + +访问示例： + +```shell +python hubserving/test_hubserving.py --server_url http://127.0.0.1:8866/predict/clas_system --image_file ./hubserving/ILSVRC2012_val_00006666.JPEG --batch_size 8 +``` + +**返回结果格式说明**： +返回结果为列表（list），包含 top-k 个分类结果，以及对应的得分，还有此图片预测耗时，具体如下： +``` +list: 返回结果 +└─ list: 第一张图片结果 + └─ list: 前 k 个分类结果，依 score 递减排序 + └─ list: 前 k 个分类结果对应的 score，依 score 递减排序 + └─ float: 该图分类耗时，单位秒 +``` + + + +## 7. 自定义修改服务模块 + +如果需要修改服务逻辑，需要进行以下操作： + +1. 停止服务 +```hub serving stop --port/-p XXXX``` + +2. 到相应的 `module.py` 和 `params.py` 等文件中根据实际需求修改代码。`module.py` 修改后需要重新安装（`hub install hubserving/clas/`）并部署。在进行部署前，可通过 `python hubserving/clas/module.py` 测试已安装服务模块。 + +3. 卸载旧服务包 +```hub uninstall clas_system``` + +4. 安装修改后的新服务包 +```hub install hubserving/clas/``` + +5.重新启动服务 +```hub serving start -m clas_system``` + +**注意**： +常用参数可在 `PaddleClas/deploy/hubserving/clas/params.py` 中修改： + * 更换模型，需要修改模型文件路径参数: + ```python + "inference_model_dir": + ``` + * 更改后处理时返回的 `top-k` 结果数量： + ```python + 'topk': + ``` + * 更改后处理时的 lable 与 class id 对应映射文件： + ```python + 'class_id_map_file': + ``` + +为了避免不必要的延时以及能够以 batch_size 进行预测，数据预处理逻辑（包括 `resize`、`crop` 等操作）均在客户端完成，因此需要在 `PaddleClas/deploy/hubserving/test_hubserving.py#L35-L52` 中修改。 diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_lite_deploy.md b/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_lite_deploy.md new file mode 100644 index 0000000..25cf5a2 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_lite_deploy.md @@ -0,0 +1,281 @@ +# PaddleLite 推理部署 +--- + +本教程将介绍基于[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite)在移动端部署 PaddleClas 分类模型的详细步骤。识别模型的部署将在近期支持，敬请期待。 + +Paddle Lite 是飞桨轻量化推理引擎，为手机、IOT 端提供高效推理能力，并广泛整合跨平台硬件，为端侧部署及应用落地问题提供轻量化的部署方案。 + +如果希望直接测试速度，可以参考[Paddle-Lite 移动端 benchmark 测试教程](../others/paddle_mobile_inference.md)。 + +--- + +## 目录 +- [1. 准备环境](#1) + - [1.1 准备交叉编译环境](#1.1) + - [1.2 准备预测库](#1.2) +- [2. 开始运行](#2) + - [2.1 模型优化](#2.1) + - [2.1.1 pip 安装 paddlelite 并进行转换](#2.1.1) + - [2.1.2 源码编译 Paddle-Lite 生成 opt 工具](#2.1.2) + - [2.1.3 转换示例](#2.1.3) + - [2.2 与手机联调](#2.2) +- [3. FAQ](#3) + + +## 1. 准备环境 + +Paddle Lite 目前支持以下平台部署： +* 电脑（编译 Paddle Lite） +* 安卓手机（armv7 或 armv8） + + +### 1.1 准备交叉编译环境 + +交叉编译环境用于编译 Paddle Lite 和 PaddleClas 的 C++ demo。 +支持多种开发环境，关于 Docker、Linux、macOS、Windows 等不同开发环境的编译流程请参考[文档](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html)。 + + +### 1.2 准备预测库 + +预测库有两种获取方式： +1. [建议]直接下载，预测库下载链接如下： + |平台|预测库下载链接| + |-|-| + |Android|[arm7](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/Android/gcc/inference_lite_lib.android.armv7.gcc.c++_static.with_extra.with_cv.tar.gz) / [arm8](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/Android/gcc/inference_lite_lib.android.armv8.gcc.c++_static.with_extra.with_cv.tar.gz)| + |iOS|[arm7](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/iOS/inference_lite_lib.ios.armv7.with_cv.with_extra.tiny_publish.tar.gz) / [arm8](https://paddlelite-data.bj.bcebos.com/Release/2.8-rc/iOS/inference_lite_lib.ios.armv8.with_cv.with_extra.tiny_publish.tar.gz)| + + **注**： + 1. 如果是从 Paddle-Lite [官方文档](https://paddle-lite.readthedocs.io/zh/latest/quick_start/release_lib.html#android-toolchain-gcc)下载的预测库， + 注意选择 `with_extra=ON，with_cv=ON` 的下载链接。 + 2. 如果使用量化的模型部署在端侧，建议使用 Paddle-Lite develop 分支编译预测库。 + +2. 编译 Paddle-Lite 得到预测库，Paddle-Lite 的编译方式如下： + +```shell +git clone https://github.com/PaddlePaddle/Paddle-Lite.git +cd Paddle-Lite +# 如果使用编译方式，建议使用 develop 分支编译预测库 +git checkout develop +./lite/tools/build_android.sh --arch=armv8 --with_cv=ON --with_extra=ON +``` +**注意**：编译 Paddle-Lite 获得预测库时，需要打开`--with_cv=ON --with_extra=ON` 两个选项，`--arch` 表示 `arm` 版本，这里指定为 armv8，更多编译命令介绍请参考[Linux x86 环境下编译适用于 Android 的库](https://paddle-lite.readthedocs.io/zh/latest/source_compile/linux_x86_compile_android.html)，关于其他平台的编译操作，具体请参考[PaddleLite](https://paddle-lite.readthedocs.io/zh/latest/)中`源码编译`部分。 + +直接下载预测库并解压后，可以得到 `inference_lite_lib.android.armv8/`文件夹，通过编译 Paddle-Lite 得到的预测库位于 `Paddle-Lite/build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/`文件夹下。 +预测库的文件目录如下： + +``` +inference_lite_lib.android.armv8/ +|-- cxx C++ 预测库和头文件 +| |-- include C++ 头文件 +| | |-- paddle_api.h +| | |-- paddle_image_preprocess.h +| | |-- paddle_lite_factory_helper.h +| | |-- paddle_place.h +| | |-- paddle_use_kernels.h +| | |-- paddle_use_ops.h +| | `-- paddle_use_passes.h +| `-- lib C++预测库 +| |-- libpaddle_api_light_bundled.a C++静态库 +| `-- libpaddle_light_api_shared.so C++动态库 +|-- java Java 预测库 +| |-- jar +| | `-- PaddlePredictor.jar +| |-- so +| | `-- libpaddle_lite_jni.so +| `-- src +|-- demo C++和Java示例代码 +| |-- cxx C++ 预测库demo +| `-- java Java 预测库demo +``` + + +## 2. 开始运行 + + +### 2.1 模型优化 + +Paddle-Lite 提供了多种策略来自动优化原始的模型，其中包括量化、子图融合、混合精度、Kernel 优选等方法，使用 Paddle-Lite 的 `opt` 工具可以自动对 inference 模型进行优化，目前支持两种优化方式，优化后的模型更轻量，模型运行速度更快。在进行模型优化前，需要先准备 `opt` 优化工具，有以下两种方式。 + +**注意**：如果已经准备好了 `.nb` 结尾的模型文件，可以跳过此步骤。 + + +#### 2.1.1 [建议]pip 安装 paddlelite 并进行转换 + +Python 下安装 `paddlelite`，目前最高支持 `Python3.7`。 +**注意**：`paddlelite` whl 包版本必须和预测库版本对应。 + +```shell +pip install paddlelite==2.8 +``` + +之后使用 `paddle_lite_opt` 工具可以进行 inference 模型的转换。`paddle_lite_opt` 的部分参数如下 + +|选项|说明| +|-|-| +|--model_dir|待优化的 PaddlePaddle 模型（非 combined 形式）的路径| +|--model_file|待优化的 PaddlePaddle 模型（combined 形式）的网络结构文件路径| +|--param_file|待优化的 PaddlePaddle 模型（combined 形式）的权重文件路径| +|--optimize_out_type|输出模型类型，目前支持两种类型：protobuf 和 naive_buffer，其中 naive_buffer 是一种更轻量级的序列化/反序列化实现。若您需要在 mobile 端执行模型预测，请将此选项设置为 naive_buffer。默认为 protobuf| +|--optimize_out|优化模型的输出路径| +|--valid_targets|指定模型可执行的 backend，默认为 arm。目前可支持 x86、arm、opencl、npu、xpu，可以同时指定多个 backend（以空格分隔），Model Optimize Tool 将会自动选择最佳方式。如果需要支持华为 NPU（Kirin 810/990 Soc 搭载的达芬奇架构 NPU），应当设置为 npu, arm| +|--record_tailoring_info|当使用 根据模型裁剪库文件 功能时，则设置该选项为 true，以记录优化后模型含有的 kernel 和 OP 信息，默认为 false| + +`--model_file` 表示 inference 模型的 model 文件地址，`--param_file` 表示 inference 模型的 param 文件地址；`optimize_out` 用于指定输出文件的名称（不需要添加 `.nb` 的后缀）。直接在命令行中运行 `paddle_lite_opt`，也可以查看所有参数及其说明。 + + +#### 2.1.2 源码编译 Paddle-Lite 生成 opt 工具 + +模型优化需要 Paddle-Lite 的 `opt` 可执行文件，可以通过编译 Paddle-Lite 源码获得，编译步骤如下： + +```shell +# 如果准备环境时已经 clone 了 Paddle-Lite，则不用重新 clone Paddle-Lite +git clone https://github.com/PaddlePaddle/Paddle-Lite.git +cd Paddle-Lite +git checkout develop +# 启动编译 +./lite/tools/build.sh build_optimize_tool +``` + +编译完成后，`opt` 文件位于 `build.opt/lite/api/` 下，可通过如下方式查看 `opt` 的运行选项和使用方式： + +```shell +cd build.opt/lite/api/ +./opt +``` + +`opt` 的使用方式与参数与上面的 `paddle_lite_opt` 完全一致。 + + +#### 2.1.3 转换示例 + +下面以 PaddleClas 的 `MobileNetV3_large_x1_0` 模型为例，介绍使用 `paddle_lite_opt` 完成预训练模型到 inference 模型，再到 Paddle-Lite 优化模型的转换。 + +```shell +# 进入 PaddleClas 根目录 +cd PaddleClas_root_path + +# 下载并解压 inference 模型 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/MobileNetV3_large_x1_0_infer.tar +tar -xf MobileNetV3_large_x1_0_infer.tar + +# 将 inference 模型转化为 Paddle-Lite 优化模型 +paddle_lite_opt --model_file=./MobileNetV3_large_x1_0_infer/inference.pdmodel --param_file=./MobileNetV3_large_x1_0_infer/inference.pdiparams --optimize_out=./MobileNetV3_large_x1_0 +``` + +最终在当前文件夹下生成 `MobileNetV3_large_x1_0.nb` 的文件。 + +**注意**：`--optimize_out` 参数为优化后模型的保存路径，无需加后缀 `.nb`；`--model_file` 参数为模型结构信息文件的路径，`--param_file` 参数为模型权重信息文件的路径，请注意文件名。 + + +### 2.2 与手机联调 + +首先需要进行一些准备工作。 +1. 准备一台 arm8 的安卓手机，如果编译的预测库和 opt 文件是 armv7，则需要 arm7 的手机，并修改 Makefile 中 `ARM_ABI = arm7`。 +2. 电脑上安装 ADB 工具，用于调试。 ADB 安装方式如下： + + * MAC 电脑安装 ADB: + ```shell + brew cask install android-platform-tools + ``` + * Linux 安装 ADB + ```shell + sudo apt update + sudo apt install -y wget adb + ``` + * Window 安装 ADB + win 上安装需要去谷歌的安卓平台下载 ADB 软件包进行安装：[链接](https://developer.android.com/studio) + +3. 手机连接电脑后，开启手机 `USB 调试` 选项，选择 `文件传输` 模式，在电脑终端中输入： + +```shell +adb devices +``` +如果有 device 输出，则表示安装成功，如下所示： +``` +List of devices attached +744be294 device +``` + +4. 准备优化后的模型、预测库文件、测试图像和类别映射文件。 + +```shell +cd PaddleClas_root_path +cd deploy/lite/ + +# 运行 prepare.sh +# prepare.sh 会将预测库文件、测试图像和使用的字典文件放置在预测库中的 demo/cxx/clas 文件夹下 +sh prepare.sh /{lite prediction library path}/inference_lite_lib.android.armv8 + +# 进入 lite demo 的工作目录 +cd /{lite prediction library path}/inference_lite_lib.android.armv8/ +cd demo/cxx/clas/ + +# 将 C++ 预测动态库 so 文件复制到 debug 文件夹中 +cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/ +``` + +`prepare.sh` 以 `PaddleClas/deploy/lite/imgs/tabby_cat.jpg` 作为测试图像，将测试图像复制到 `demo/cxx/clas/debug/` 文件夹下。 +将 `paddle_lite_opt` 工具优化后的模型文件放置到 `/{lite prediction library path}/inference_lite_lib.android.armv8/demo/cxx/clas/debug/` 文件夹下。本例中，使用 [2.1.3 转换示例](#2.1.3) 生成的 `MobileNetV3_large_x1_0.nb` 模型文件。 + +执行完成后，clas 文件夹下将有如下文件格式： + +``` +demo/cxx/clas/ +|-- debug/ +| |--MobileNetV3_large_x1_0.nb 优化后的分类器模型文件 +| |--tabby_cat.jpg 待测试图像 +| |--imagenet1k_label_list.txt 类别映射文件 +| |--libpaddle_light_api_shared.so C++预测库文件 +| |--config.txt 分类预测超参数配置 +|-- config.txt 分类预测超参数配置 +|-- image_classfication.cpp 图像分类代码文件 +|-- Makefile 编译文件 +``` + +#### 注意： +* 上述文件中，`imagenet1k_label_list.txt` 是 ImageNet1k 数据集的类别映射文件，如果使用自定义的类别，需要更换该类别映射文件。 + +* `config.txt` 包含了分类器的超参数，如下： + +```shell +clas_model_file ./MobileNetV3_large_x1_0.nb # 模型文件地址 +label_path ./imagenet1k_label_list.txt # 类别映射文本文件 +resize_short_size 256 # resize 之后的短边边长 +crop_size 224 # 裁剪后用于预测的边长 +visualize 0 # 是否进行可视化，如果选择的话，会在当前文件夹下生成名为 clas_result.png 的图像文件。 +``` + +5. 启动调试，上述步骤完成后就可以使用 ADB 将文件夹 `debug/` push 到手机上运行，步骤如下： + +```shell +# 执行编译，得到可执行文件 clas_system +make -j + +# 将编译得到的可执行文件移动到 debug 文件夹中 +mv clas_system ./debug/ + +# 将上述 debug 文件夹 push 到手机上 +adb push debug /data/local/tmp/ + +adb shell +cd /data/local/tmp/debug +export LD_LIBRARY_PATH=/data/local/tmp/debug:$LD_LIBRARY_PATH + +# clas_system 可执行文件的使用方式为: +# ./clas_system 配置文件路径 测试图像路径 +./clas_system ./config.txt ./tabby_cat.jpg +``` + +如果对代码做了修改，则需要重新编译并 push 到手机上。 + +运行效果如下： + +![](../../images/inference_deployment/lite_demo_result.png) + + +## 3. FAQ +Q1：如果想更换模型怎么办，需要重新按照流程走一遍吗？ +A1：如果已经走通了上述步骤，更换模型只需要替换 `.nb` 模型文件即可，同时要注意修改下配置文件中的 `.nb` 文件路径以及类别映射文件（如有必要）。 + +Q2：换一个图测试怎么做？ +A2：替换 debug 下的测试图像为你想要测试的图像，使用 ADB 再次 push 到手机上即可。 diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_serving_deploy.md b/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_serving_deploy.md new file mode 100644 index 0000000..18faeb3 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/paddle_serving_deploy.md @@ -0,0 +1,294 @@ +# 模型服务化部署 +-------- +## 目录 +- [1. 简介](#1) +- [2. Serving 安装](#2) +- [3. 图像分类服务部署](#3) + - [3.1 模型转换](#3.1) + - [3.2 服务部署和请求](#3.2) + - [3.2.1 Python Serving](#3.2.1) + - [3.2.2 C++ Serving](#3.2.2) +- [4. 图像识别服务部署](#4) + - [4.1 模型转换](#4.1) + - [4.2 服务部署和请求](#4.2) + - [4.2.1 Python Serving](#4.2.1) + - [4.2.2 C++ Serving](#4.2.2) +- [5. FAQ](#5) + + +## 1. 简介 +[Paddle Serving](https://github.com/PaddlePaddle/Serving) 旨在帮助深度学习开发者轻松部署在线预测服务，支持一键部署工业级的服务能力、客户端和服务端之间高并发和高效通信、并支持多种编程语言开发客户端。 + +该部分以 HTTP 预测服务部署为例，介绍怎样在 PaddleClas 中使用 PaddleServing 部署模型服务。目前只支持 Linux 平台部署，暂不支持 Windows 平台。 + + +## 2. Serving 安装 +Serving 官网推荐使用 docker 安装并部署 Serving 环境。首先需要拉取 docker 环境并创建基于 Serving 的 docker。 + +```shell +# 启动GPU docker +docker pull paddlepaddle/serving:0.7.0-cuda10.2-cudnn7-devel +nvidia-docker run -p 9292:9292 --name test -dit paddlepaddle/serving:0.7.0-cuda10.2-cudnn7-devel bash +nvidia-docker exec -it test bash + +# 启动CPU docker +docker pull paddlepaddle/serving:0.7.0-devel +docker run -p 9292:9292 --name test -dit paddlepaddle/serving:0.7.0-devel bash +docker exec -it test bash +``` + +进入 docker 后，需要安装 Serving 相关的 python 包。 +```shell +pip3 install paddle-serving-client==0.7.0 +pip3 install paddle-serving-app==0.7.0 +pip3 install faiss-cpu==1.7.1post2 + +#若为CPU部署环境: +pip3 install paddle-serving-server==0.7.0 # CPU +pip3 install paddlepaddle==2.2.0 # CPU + +#若为GPU部署环境 +pip3 install paddle-serving-server-gpu==0.7.0.post102 # GPU with CUDA10.2 + TensorRT6 +pip3 install paddlepaddle-gpu==2.2.0 # GPU with CUDA10.2 + +#其他GPU环境需要确认环境再选择执行哪一条 +pip3 install paddle-serving-server-gpu==0.7.0.post101 # GPU with CUDA10.1 + TensorRT6 +pip3 install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUDA11.2 + TensorRT8 +``` + +* 如果安装速度太慢，可以通过 `-i https://pypi.tuna.tsinghua.edu.cn/simple` 更换源，加速安装过程。 +* 其他环境配置安装请参考: [使用Docker安装Paddle Serving](https://github.com/PaddlePaddle/Serving/blob/v0.7.0/doc/Install_CN.md) + + + +## 3. 图像分类服务部署 + +### 3.1 模型转换 +使用 PaddleServing 做服务化部署时，需要将保存的 inference 模型转换为 Serving 模型。下面以经典的 ResNet50_vd 模型为例，介绍如何部署图像分类服务。 +- 进入工作目录： +```shell +cd deploy/paddleserving +``` +- 下载 ResNet50_vd 的 inference 模型： +```shell +# 下载并解压 ResNet50_vd 模型 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/ResNet50_vd_infer.tar && tar xf ResNet50_vd_infer.tar +``` +- 用 paddle_serving_client 把下载的 inference 模型转换成易于 Server 部署的模型格式： +``` +# 转换 ResNet50_vd 模型 +python3 -m paddle_serving_client.convert --dirname ./ResNet50_vd_infer/ \ + --model_filename inference.pdmodel \ + --params_filename inference.pdiparams \ + --serving_server ./ResNet50_vd_serving/ \ + --serving_client ./ResNet50_vd_client/ +``` +ResNet50_vd 推理模型转换完成后，会在当前文件夹多出 `ResNet50_vd_serving` 和 `ResNet50_vd_client` 的文件夹，具备如下格式： +``` +|- ResNet50_vd_serving/ + |- inference.pdiparams + |- inference.pdmodel + |- serving_server_conf.prototxt + |- serving_server_conf.stream.prototxt +|- ResNet50_vd_client + |- serving_client_conf.prototxt + |- serving_client_conf.stream.prototxt +``` +得到模型文件之后，需要分别修改 `ResNet50_vd_serving` 和 `ResNet50_vd_client` 下文件 `serving_server_conf.prototxt` 中的 alias 名字：将 `fetch_var` 中的 `alias_name` 改为 `prediction` + +**备注**: Serving 为了兼容不同模型的部署，提供了输入输出重命名的功能。这样，不同的模型在推理部署时，只需要修改配置文件的 alias_name 即可，无需修改代码即可完成推理部署。 +修改后的 serving_server_conf.prototxt 如下所示: +``` +feed_var { + name: "inputs" + alias_name: "inputs" + is_lod_tensor: false + feed_type: 1 + shape: 3 + shape: 224 + shape: 224 +} +fetch_var { + name: "save_infer_model/scale_0.tmp_1" + alias_name: "prediction" + is_lod_tensor: false + fetch_type: 1 + shape: 1000 +} +``` + +### 3.2 服务部署和请求 +paddleserving 目录包含了启动 pipeline 服务、C++ serving服务和发送预测请求的代码，包括： +```shell +__init__.py +config.yml # 启动pipeline服务的配置文件 +pipeline_http_client.py # http方式发送pipeline预测请求的脚本 +pipeline_rpc_client.py # rpc方式发送pipeline预测请求的脚本 +classification_web_service.py # 启动pipeline服务端的脚本 +run_cpp_serving.sh # 启动C++ Serving部署的脚本 +test_cpp_serving_client.py # rpc方式发送C++ serving预测请求的脚本 +``` + +#### 3.2.1 Python Serving +- 启动服务： +```shell +# 启动服务，运行日志保存在 log.txt +python3 classification_web_service.py &>log.txt & +``` + +- 发送请求： +```shell +# 发送服务请求 +python3 pipeline_http_client.py +``` +成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下： +``` +{'err_no': 0, 'err_msg': '', 'key': ['label', 'prob'], 'value': ["['daisy']", '[0.9341402053833008]'], 'tensors': []} +``` + + +#### 3.2.2 C++ Serving +- 启动服务： +```shell +# 启动服务， 服务在后台运行，运行日志保存在 nohup.txt +sh run_cpp_serving.sh +``` + +- 发送请求： +```shell +# 发送服务请求 +python3 test_cpp_serving_client.py +``` +成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下： +``` +prediction: daisy, probability: 0.9341399073600769 +``` + + +## 4.图像识别服务部署 +使用 PaddleServing 做服务化部署时，需要将保存的 inference 模型转换为 Serving 模型。 下面以 PP-ShiTu 中的超轻量图像识别模型为例，介绍图像识别服务的部署。 + +## 4.1 模型转换 +- 下载通用检测 inference 模型和通用识别 inference 模型 +``` +cd deploy +# 下载并解压通用识别模型 +wget -P models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar +cd models +tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar +# 下载并解压通用检测模型 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar +tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar +``` +- 转换识别 inference 模型为 Serving 模型： +``` +# 转换识别模型 +python3 -m paddle_serving_client.convert --dirname ./general_PPLCNet_x2_5_lite_v1.0_infer/ \ + --model_filename inference.pdmodel \ + --params_filename inference.pdiparams \ + --serving_server ./general_PPLCNet_x2_5_lite_v1.0_serving/ \ + --serving_client ./general_PPLCNet_x2_5_lite_v1.0_client/ +``` +识别推理模型转换完成后，会在当前文件夹多出 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 的文件夹。分别修改 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 目录下的 serving_server_conf.prototxt 中的 alias 名字： 将 `fetch_var` 中的 `alias_name` 改为 `features`。 +修改后的 serving_server_conf.prototxt 内容如下： +``` +feed_var { + name: "x" + alias_name: "x" + is_lod_tensor: false + feed_type: 1 + shape: 3 + shape: 224 + shape: 224 +} +fetch_var { + name: "save_infer_model/scale_0.tmp_1" + alias_name: "features" + is_lod_tensor: false + fetch_type: 1 + shape: 512 +} +``` +- 转换通用检测 inference 模型为 Serving 模型： +``` +# 转换通用检测模型 +python3 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \ + --model_filename inference.pdmodel \ + --params_filename inference.pdiparams \ + --serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \ + --serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/ +``` +检测 inference 模型转换完成后，会在当前文件夹多出 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` 和 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/` 的文件夹。 + +**注意:** 此处不需要修改 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` 目录下的 serving_server_conf.prototxt 中的 alias 名字。 + +- 下载并解压已经构建后的检索库 index +``` +cd ../ +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar && tar -xf drink_dataset_v1.0.tar +``` + +## 4.2 服务部署和请求 +**注意:** 识别服务涉及到多个模型，出于性能考虑采用 PipeLine 部署方式。Pipeline 部署方式当前不支持 windows 平台。 +- 进入到工作目录 +```shell +cd ./deploy/paddleserving/recognition +``` +paddleserving 目录包含启动 Python Pipeline 服务、C++ Serving 服务和发送预测请求的代码，包括： +``` +__init__.py +config.yml # 启动python pipeline服务的配置文件 +pipeline_http_client.py # http方式发送pipeline预测请求的脚本 +pipeline_rpc_client.py # rpc方式发送pipeline预测请求的脚本 +recognition_web_service.py # 启动pipeline服务端的脚本 +run_cpp_serving.sh # 启动C++ Pipeline Serving部署的脚本 +test_cpp_serving_client.py # rpc方式发送C++ Pipeline serving预测请求的脚本 +``` + + +#### 4.2.1 Python Serving +- 启动服务： +``` +# 启动服务，运行日志保存在 log.txt +python3 recognition_web_service.py &>log.txt & +``` + +- 发送请求： +``` +python3 pipeline_http_client.py +``` +成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下： +``` +{'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["[{'bbox': [345, 95, 524, 576], 'rec_docs': '红牛-强化型', 'rec_scores': 0.79903316}]"], 'tensors': []} +``` + + +#### 4.2.2 C++ Serving +- 启动服务： +```shell +# 启动服务： 此处会在后台同时启动主体检测和特征提取服务，端口号分别为9293和9294； +# 运行日志分别保存在 log_mainbody_detection.txt 和 log_feature_extraction.txt中 +sh run_cpp_serving.sh +``` + +- 发送请求： +```shell +# 发送服务请求 +python3 test_cpp_serving_client.py +``` +成功运行后，模型预测的结果会打印在 cmd 窗口中，结果如下所示： +``` +[{'bbox': [345, 95, 524, 586], 'rec_docs': '红牛-强化型', 'rec_scores': 0.8016462}] +``` + + +## 5.FAQ +**Q1**： 发送请求后没有结果返回或者提示输出解码报错 + +**A1**： 启动服务和发送请求时不要设置代理，可以在启动服务前和发送请求前关闭代理，关闭代理的命令是： +``` +unset https_proxy +unset http_proxy +``` + +更多的服务部署类型，如 `RPC 预测服务` 等，可以参考 Serving 的[github 官网](https://github.com/PaddlePaddle/Serving/tree/v0.7.0/examples) diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/python_deploy.md b/src/PaddleClas/docs/zh_CN/inference_deployment/python_deploy.md new file mode 100644 index 0000000..39843df --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/python_deploy.md @@ -0,0 +1,96 @@ +# Python 预测推理 + +--- + +首先请参考文档[安装 PaddlePaddle](../installation/install_paddle.md)和文档[安装 PaddleClas](../installation/install_paddleclas.md)配置运行环境。 + +## 目录 + +- [1. 图像分类推理](#1) +- [2. 主体检测模型推理](#2) +- [3. 特征提取模型推理](#3) +- [4. 主体检测、特征提取和向量检索串联](#4) + + +## 1. 图像分类推理 + +首先请参考文档[模型导出](./export_model.md)准备 inference 模型，然后进入 PaddleClas 的 `deploy` 目录下： + +```shell +cd /path/to/PaddleClas/deploy +``` + +使用以下命令进行预测： + +```shell +python python/predict_cls.py -c configs/inference_cls.yaml +``` + +在配置文件 `configs/inference_cls.yaml` 中有以下字段用于配置预测参数： +* `Global.infer_imgs`：待预测的图片文件路径； +* `Global.inference_model_dir`：inference 模型文件所在目录，该目录下需要有文件 `inference.pdmodel` 和 `inference.pdiparams` 两个文件； +* `Global.use_tensorrt`：是否使用 TesorRT 预测引擎，默认为 `False`； +* `Global.use_gpu`：是否使用 GPU 预测，默认为 `True`； +* `Global.enable_mkldnn`：是否启用 `MKL-DNN` 加速库，默认为 `False`。注意 `enable_mkldnn` 与 `use_gpu` 同时为 `True` 时，将忽略 `enable_mkldnn`，而使用 GPU 预测； +* `Global.use_fp16`：是否启用 `FP16`，默认为 `False`； +* `PreProcess`：用于数据预处理配置； +* `PostProcess`：由于后处理配置； +* `PostProcess.Topk.class_id_map_file`：数据集 label 的映射文件，默认为 `./utils/imagenet1k_label_list.txt`，该文件为 PaddleClas 所使用的 ImageNet 数据集 label 映射文件。 + +**注意**: +* 如果使用 VisionTransformer 系列模型，如 `DeiT_***_384`, `ViT_***_384` 等，请注意模型的输入数据尺寸，部分模型需要修改参数： `PreProcess.resize_short=384`, `PreProcess.resize=384`。 +* 如果你希望提升评测模型速度，使用 GPU 评测时，建议开启 TensorRT 加速预测，使用 CPU 评测时，建议开启 MKL-DNN 加速预测。 + + +## 2. 主体检测模型推理 + +进入 PaddleClas 的 `deploy` 目录下： + +```shell +cd /path/to/PaddleClas/deploy +``` + +准备 PaddleClas 提供的主体检测 inference 模型： + +```shell +mkdir -p models +# 下载通用检测 inference 模型并解压 +wget -P ./models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar +tar -xf ./models/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar -C ./models/ +``` + +使用以下命令进行预测： + +```shell +python python/predict_det.py -c configs/inference_det.yaml +``` + +在配置文件 `configs/inference_det.yaml` 中有以下字段用于配置预测参数： +* `Global.infer_imgs`：待预测的图片文件路径； +* `Global.use_gpu`： 是否使用 GPU 预测，默认为 `True`。 + + + +## 3. 特征提取模型推理 + +下面以商品特征提取为例，介绍特征提取模型推理。首先进入 PaddleClas 的 `deploy` 目录下： + +```shell +cd /path/to/PaddleClas/deploy +``` + +准备 PaddleClas 提供的商品特征提取 inference 模型： + +```shell +mkdir -p models +# 下载商品特征提取 inference 模型并解压 +wget -P ./models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar +tar -xf ./models/product_ResNet50_vd_aliproduct_v1.0_infer.tar -C ./models/ +``` + +上述预测命令可以得到一个 512 维的特征向量，直接输出在在命令行中。 + + +## 4. 主体检测、特征提取和向量检索串联 + +主体检测、特征提取和向量检索的串联预测，可以参考图像识别[快速体验](../quick_start/quick_start_recognition.md)。 diff --git a/src/PaddleClas/docs/zh_CN/inference_deployment/whl_deploy.md b/src/PaddleClas/docs/zh_CN/inference_deployment/whl_deploy.md new file mode 100644 index 0000000..14582ac --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/inference_deployment/whl_deploy.md @@ -0,0 +1,256 @@ +# PaddleClas Whl 使用说明 + +PaddleClas 支持 Python Whl 包方式进行预测，目前 Whl 包方式仅支持图像分类，暂不支持主体检测、特征提取及向量检索。 + +--- + +## 目录 + +- [1. 安装 paddleclas](#1) +- [2. 快速开始](#2) +- [3. 参数解释](#3) +- [4. 使用示例](#4) + - [4.1 查看帮助信息](#4.1) + - [4.2 使用 PaddleClas 提供的预训练模型进行预测](#4.2) + - [4.3 使用本地模型文件预测](#4.3) + - [4.4 批量预测](#4.4) + - [4.5 对网络图片进行预测](#4.5) + - [4.6 对 `NumPy.ndarray` 格式数据进行预测](#4.6) + - [4.7 保存预测结果](#4.7) + - [4.8 指定 label name](#4.8) + + + +## 1. 安装 paddleclas + +* pip 安装 + +```bash +pip3 install paddleclas==2.2.1 +``` + +* 本地构建并安装 + +```bash +python3 setup.py bdist_wheel +pip3 install dist/* +``` + + +## 2. 快速开始 +* 使用 `ResNet50` 模型，以下图（`PaddleClas/docs/images/inference_deployment/whl_demo.jpg`）为例进行说明。 + +![](../../images/inference_deployment/whl_demo.jpg) + + +* 在 Python 代码中使用 +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs='docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +**注意**：`PaddleClas.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果。返回结果示例如下： + +``` +>>> result +[{'class_ids': [8, 7, 136, 80, 84], 'scores': [0.79368, 0.16329, 0.01853, 0.00959, 0.00239], 'label_names': ['hen', 'cock', 'European gallinule, Porphyrio porphyrio', 'black grouse', 'peacock']}] +``` + +* 在命令行中使用 +```bash +paddleclas --model_name=ResNet50 --infer_imgs="docs/images/inference_deployment/whl_demo.jpg" +``` + +``` +>>> result +filename: docs/images/inference_deployment/whl_demo.jpg, top-5, class_ids: [8, 7, 136, 80, 84], scores: [0.79368, 0.16329, 0.01853, 0.00959, 0.00239], label_names: ['hen', 'cock', 'European gallinule, Porphyrio porphyrio', 'black grouse', 'peacock'] +Predict complete! +``` + + +## 3. 参数解释 +以下参数可在命令行方式使用中通过参数指定，或在 Python 代码中实例化 PaddleClas 对象时作为构造函数的参数使用。 +* model_name(str): 模型名称，使用 PaddleClas 提供的基于 ImageNet1k 的预训练模型。 +* inference_model_dir(str): 本地模型文件目录，当未指定 `model_name` 时该参数有效。该目录下需包含 `inference.pdmodel` 和 `inference.pdiparams` 两个模型文件。 +* infer_imgs(str): 待预测图片文件路径，或包含图片文件的目录，或网络图片的 URL。 +* use_gpu(bool): 是否使用 GPU，默认为 `True`。 +* gpu_mem(int): 使用的 GPU 显存大小，当 `use_gpu` 为 `True` 时有效，默认为 8000。 +* use_tensorrt(bool): 是否开启 TensorRT 预测，可提升 GPU 预测性能，需要使用带 TensorRT 的预测库，默认为 `False`。 +* enable_mkldnn(bool): 是否开启 MKLDNN，当 `use_gpu` 为 `False` 时有效，默认 `False`。 +* cpu_num_threads(int): CPU 预测时的线程数，当 `use_gpu` 为 `False` 且 `enable_mkldnn` 为 `True` 时有效，默认值为 `10`。 +* batch_size(int): 预测时每个 batch 的样本数量，默认为 `1`。 +* resize_short(int): 按图像较短边进行等比例缩放，默认为 `256`。 +* crop_size(int): 将图像裁剪到指定大小，默认为 `224`。 +* topk(int): 打印（返回）预测结果的前 `topk` 个类别和对应的分类概率，默认为 `5`。 +* class_id_map_file(str): `class id` 与 `label` 的映射关系文件。默认使用 `ImageNet1K` 数据集的映射关系。 +* save_dir(str): 将预测结果作为预标注数据保存的路径，默认为 `None`，即不保存。 + +**注意**: 如果使用 `Transformer` 系列模型，如 `DeiT_***_384`, `ViT_***_384` 等，请注意模型的输入数据尺寸，需要设置参数 `resize_short=384`, `crop_size=384`，如下所示。 + +* 命令行中 +```bash +from paddleclas import PaddleClas, get_default_confg +paddleclas --model_name=ViT_base_patch16_384 --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' --resize_short=384 --crop_size=384 +``` + +* Python 代码中 +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ViT_base_patch16_384', resize_short=384, crop_size=384) +``` + + +## 4. 使用示例 + +PaddleClas 提供两种使用方式： +1. Python 代码中使用； +2. 命令行中使用。 + + +### 4.1 查看帮助信息 + +* CLI +```bash +paddleclas -h +``` + + +### 4.2 使用 PaddleClas 提供的预训练模型进行预测 +可以使用 PaddleClas 提供的预训练模型来预测，并通过参数 `model_name` 指定。此时 PaddleClas 会根据 `model_name` 自动下载指定模型，并保存在目录`~/.paddleclas/`下。 + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs = 'docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' +``` + + +### 4.3 使用本地模型文件预测 +可以使用本地的模型文件进行预测，通过参数 `inference_model_dir` 指定模型文件目录即可。需要注意，模型文件目录下必须包含 `inference.pdmodel` 和 `inference.pdiparams` 两个文件。 + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(inference_model_dir='./inference/') +infer_imgs = 'docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --inference_model_dir='./inference/' --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' +``` + + +### 4.4 批量预测 +当参数 `infer_imgs` 为包含图片文件的目录时，可以对图片进行批量预测，只需通过参数 `batch_size` 指定 batch 大小。 + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50', batch_size=2) +infer_imgs = 'docs/images/' +result=clas.predict(infer_imgs) +for r in result: + print(r) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/' --batch_size 2 +``` + + +### 4.5 对网络图片进行预测 +可以对网络图片进行预测，只需通过参数 `infer_imgs` 指定图片 `url`。此时图片会下载并保存在`~/.paddleclas/images/`目录下。 + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs = 'https://raw.githubusercontent.com/paddlepaddle/paddleclas/release/2.2/docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='https://raw.githubusercontent.com/paddlepaddle/paddleclas/release/2.2/docs/images/inference_deployment/whl_demo.jpg' +``` + + +### 4.6 对 `NumPy.ndarray` 格式数据进行预测 +在 Python 中，可以对 `Numpy.ndarray` 格式的图像数据进行预测，只需通过参数 `infer_imgs` 指定即可。注意，PaddleClas 所提供的模型仅支持 3 通道图像数据，且通道顺序为 `RGB`。 + +* python +```python +import cv2 +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50') +infer_imgs = cv2.imread("docs/images/inference_deployment/whl_demo.jpg")[:, :, ::-1] +result=clas.predict(infer_imgs) +print(next(result)) +``` + + +### 4.7 保存预测结果 +可以指定参数 `pre_label_out_dir='./output_pre_label/'`，将图片按其 top1 预测结果保存到 `pre_label_out_dir` 目录下对应类别的文件夹中。 + +* python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50', save_dir='./output_pre_label/') +infer_imgs = 'docs/images/whl/' # it can be infer_imgs folder path which contains all of images you want to predict. +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/whl/' --save_dir='./output_pre_label/' +``` + + +### 4.8 指定 label name +可以通过参数 `class_id_map_file` 指定 `class id` 与 `lable` 的对应关系。PaddleClas 默认使用 ImageNet1K 的 label_name（`ppcls/utils/imagenet1k_label_list.txt`）。 + +`class_id_map_file` 文件内容格式应为： + +``` +class_idclass_name<\n> +``` + +例如： + +``` +0 tench, Tinca tinca +1 goldfish, Carassius auratus +2 great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias +...... +``` + +* Python +```python +from paddleclas import PaddleClas +clas = PaddleClas(model_name='ResNet50', class_id_map_file='./ppcls/utils/imagenet1k_label_list.txt') +infer_imgs = 'docs/images/inference_deployment/whl_demo.jpg' +result=clas.predict(infer_imgs) +print(next(result)) +``` + +* CLI +```bash +paddleclas --model_name='ResNet50' --infer_imgs='docs/images/inference_deployment/whl_demo.jpg' --class_id_map_file='./ppcls/utils/imagenet1k_label_list.txt' +``` diff --git a/src/PaddleClas/docs/zh_CN/installation/index.rst b/src/PaddleClas/docs/zh_CN/installation/index.rst new file mode 100644 index 0000000..752d050 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/installation/index.rst @@ -0,0 +1,8 @@ +安装 +================================ + +.. toctree:: + :maxdepth: 2 + + install_paddle.md + install_paddleclas.md diff --git a/src/PaddleClas/docs/zh_CN/installation/install_paddle.md b/src/PaddleClas/docs/zh_CN/installation/install_paddle.md new file mode 100644 index 0000000..995d287 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/installation/install_paddle.md @@ -0,0 +1,101 @@ +# 安装 PaddlePaddle + +--- +## 目录 + +- [1. 环境要求](#1) +- [2.（建议）使用 Docker 环境](#2) +- [3. 通过 pip 安装 PaddlePaddle](#3) +- [4. 验证安装](#4) + +目前，**PaddleClas** 要求 **PaddlePaddle** 版本 `>=2.0`。建议使用我们提供的 Docker 运行 PaddleClas，有关 Docker、nvidia-docker 的相关使用教程可以参考[链接](https://www.runoob.com/Docker/Docker-tutorial.html)。如果不使用 Docker，可以直接跳过 [2.（建议）使用 Docker 环境](#2) 部分内容，从 [3. 通过 pip 安装 PaddlePaddle](#3) 部分开始。 + + + +## 1. 环境要求 + +**版本要求**： +- python 3.x +- CUDA >= 10.1（如果使用 `paddlepaddle-gpu`） +- cuDNN >= 7.6.4（如果使用 `paddlepaddle-gpu`） +- nccl >= 2.1.2（如果使用分布式训练/评估） +- gcc >= 8.2 + +**建议**： +* 当 CUDA 版本为 10.1 时，显卡驱动版本 `>= 418.39`； +* 当 CUDA 版本为 10.2 时，显卡驱动版本 `>= 440.33`； +* 更多 CUDA 版本与要求的显卡驱动版本可以参考[链接](https://docs.nvidia.com/deploy/cuda-compatibility/index.html)。 + + + +## 2.（建议）使用 Docker 环境 + +* 切换到工作目录下 + +```shell +cd /home/Projects +``` + +* 创建 docker 容器 + +下述命令会创建一个名为 ppcls 的 Docker 容器，并将当前工作目录映射到容器内的 `/paddle` 目录。 + +```shell +# 对于 GPU 用户 +sudo nvidia-docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it paddlepaddle/paddle:2.1.0-gpu-cuda10.2-cudnn7 /bin/bash + +# 对于 CPU 用户 +sudo docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it paddlepaddle/paddle:2.1.0 /bin/bash +``` + +**注意**： +* 首次使用该镜像时，下述命令会自动下载该镜像文件，下载需要一定的时间，请耐心等待； +* 上述命令会创建一个名为 ppcls 的 Docker 容器，之后再次使用该容器时无需再次运行该命令； +* 参数 `--shm-size=8G` 将设置容器的共享内存为 8 G，如机器环境允许，建议将该参数设置较大，如 `64G`； +* 您也可以访问 [DockerHub](https://hub.Docker.com/r/paddlepaddle/paddle/tags/) 获取与您机器适配的镜像； +* 退出/进入 docker 容器： + * 在进入 Docker 容器后，可使用组合键 `Ctrl + P + Q` 退出当前容器，同时不关闭该容器； + * 如需再次进入容器，可使用下述命令： + + ```shell + sudo Docker exec -it ppcls /bin/bash + ``` + + + +## 3. 通过 pip 安装 PaddlePaddle + +可运行下面的命令，通过 pip 安装最新版本 PaddlePaddle： + +```bash +# 对于 CPU 用户 +pip install paddlepaddle --upgrade -i https://mirror.baidu.com/pypi/simple + +# 对于 GPU 用户 +pip install paddlepaddle-gpu --upgrade -i https://mirror.baidu.com/pypi/simple +``` + +**注意：** +* 如果先安装了 CPU 版本的 PaddlePaddle，之后想切换到 GPU 版本，那么需要使用 pip 先卸载 CPU 版本的 PaddlePaddle，再安装 GPU 版本的 PaddlePaddle，否则容易导致 PaddlePaddle 冲突。 +* 您也可以从源码编译安装 PaddlePaddle，请参照 [PaddlePaddle 安装文档](http://www.paddlepaddle.org.cn/install/quick) 中的说明进行操作。 + + +## 4. 验证安装 + +使用以下命令可以验证 PaddlePaddle 是否安装成功。 + +```python +import paddle +paddle.utils.run_check() +``` + +查看 PaddlePaddle 版本的命令如下： + +```bash +python -c "import paddle; print(paddle.__version__)" +``` + +**注意**： +- 从源码编译的 PaddlePaddle 版本号为 `0.0.0`，请确保使用 PaddlePaddle 2.0 及之后的源码进行编译； +- PaddleClas 基于 PaddlePaddle 高性能的分布式训练能力，若您从源码编译，请确保打开编译选项 `WITH_DISTRIBUTE=ON`。具体编译选项参考 [编译选项表](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#bianyixuanxiangbiao)； +- 在 Docker 中运行时，为保证 Docker 容器有足够的共享内存用于 Paddle 的数据读取加速，在创建 Docker 容器时，请设置参数 `--shm-size=8g`，条件允许的话可以设置为更大的值。 diff --git a/src/PaddleClas/docs/zh_CN/installation/install_paddleclas.md b/src/PaddleClas/docs/zh_CN/installation/install_paddleclas.md new file mode 100644 index 0000000..0f70bf2 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/installation/install_paddleclas.md @@ -0,0 +1,32 @@ +# 安装 PaddleClas + +--- +## 目录 + +* [1. 克隆 PaddleClas](#1) +* [2. 安装 Python 依赖库](#2) + + + +## 1. 克隆 PaddleClas + +从 GitHub 下载： + +```shell +git clone https://github.com/PaddlePaddle/PaddleClas.git -b release/2.3 +``` + +如果访问 GitHub 网速较慢，可以从 Gitee 下载，命令如下： + +```shell +git clone https://gitee.com/paddlepaddle/PaddleClas.git -b release/2.3 +``` + + +## 2. 安装 Python 依赖库 + +PaddleClas 的 Python 依赖库在 `requirements.txt` 中给出，可通过如下命令安装： + +```shell +pip install --upgrade -r requirements.txt -i https://mirror.baidu.com/pypi/simple +``` diff --git a/src/PaddleClas/docs/zh_CN/introduction/function_intro.md b/src/PaddleClas/docs/zh_CN/introduction/function_intro.md new file mode 100644 index 0000000..63afc75 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/function_intro.md @@ -0,0 +1,20 @@ +## PaddleClas 功能特性 + +飞桨图像识别套件 PaddleClas 是飞桨为工业界和学术界所准备的一个图像识别任务的工具集，助力使用者训练出更好的视觉模型和应用落地。具体地，它包含以下几个核心特性。 + + +- 实用的图像识别系统：集成了目标检测、特征学习、图像检索等模块，广泛适用于各类图像识别任务。 +提供商品识别、车辆识别、 logo 识别和动漫人物识别等 4 个场景应用示例。 + +- 丰富的预训练模型库：提供了 36 个系列共 175 个 ImageNet 预训练模型，其中 7 个精选系列模型支持结构快速修改。 + +- 全面易用的特征学习组件：集成 arcmargin, triplet loss 等 12 度量学习方法，通过配置文件即可随意组合切换。 + +- SSLD 知识蒸馏： 14 个分类预训练模型，精度普遍提升 3% 以上；其中 ResNet50_vd 模型在 ImageNet-1k 数据集上的 Top-1 精度达到了 84.0%，Res2Net200_vd 预训练模型 Top-1 精度高达 85.1% 。 + +- 数据增广：支持 AutoAugment 、 Cutout 、 Cutmix 等 8 种数据增广算法详细介绍、代码复现和在统一实验环境下的效果评估。 + + +![](../../images/recognition.gif) + +更多关于图像识别快速体验、算法详解、模型训练评估与预测部署方法，请参考[首页 README 文档教程](../../../README_ch.md)。 diff --git a/src/PaddleClas/docs/zh_CN/introduction/index.rst b/src/PaddleClas/docs/zh_CN/introduction/index.rst new file mode 100644 index 0000000..c33c689 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/index.rst @@ -0,0 +1,8 @@ +介绍 +================================ + +.. toctree:: + :maxdepth: 2 + + function_intro.md + more_demo/index diff --git a/src/PaddleClas/docs/zh_CN/introduction/more_demo/cartoon.md b/src/PaddleClas/docs/zh_CN/introduction/more_demo/cartoon.md new file mode 100644 index 0000000..825da49 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/more_demo/cartoon.md @@ -0,0 +1,53 @@ +## 动漫人物图片识别效果图 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/zh_CN/introduction/more_demo/index.rst b/src/PaddleClas/docs/zh_CN/introduction/more_demo/index.rst new file mode 100644 index 0000000..448e9fe --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/more_demo/index.rst @@ -0,0 +1,11 @@ +介绍 +================================ + +.. toctree:: + :maxdepth: 2 + + cartoon.md + logo.md + more_demo.md + product.md + vehicle.md diff --git a/src/PaddleClas/docs/zh_CN/introduction/more_demo/logo.md b/src/PaddleClas/docs/zh_CN/introduction/more_demo/logo.md new file mode 100644 index 0000000..cbed4a6 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/more_demo/logo.md @@ -0,0 +1,65 @@ +## 商标图片识别效果图 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/zh_CN/introduction/more_demo/more_demo.md b/src/PaddleClas/docs/zh_CN/introduction/more_demo/more_demo.md new file mode 100644 index 0000000..97732e3 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/more_demo/more_demo.md @@ -0,0 +1,40 @@ +## 识别效果展示 +- 商品识别 +
+
+
+
+
+ +[更多效果图](product.md) + + +- 动漫人物识别 + +
+
+
+ +[更多效果图](cartoon.md) + + +- logo识别 + +
+
+
+
+
+
+ + +[更多效果图](logo.md) + + +- 车辆识别 + +
+
+
+ +[更多效果图](vehicle.md) diff --git a/src/PaddleClas/docs/zh_CN/introduction/more_demo/product.md b/src/PaddleClas/docs/zh_CN/introduction/more_demo/product.md new file mode 100644 index 0000000..df211f0 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/more_demo/product.md @@ -0,0 +1,179 @@ +## 商品图片识别效果图 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/zh_CN/introduction/more_demo/vehicle.md b/src/PaddleClas/docs/zh_CN/introduction/more_demo/vehicle.md new file mode 100644 index 0000000..975c98b --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/introduction/more_demo/vehicle.md @@ -0,0 +1,33 @@ +## 车辆图片识别效果图 + +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
diff --git a/src/PaddleClas/docs/zh_CN/make.bat b/src/PaddleClas/docs/zh_CN/make.bat new file mode 100644 index 0000000..ed767fc --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/make.bat @@ -0,0 +1,37 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) + +set SOURCEDIR=source +set BUILDDIR=build + + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.https://www.sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/src/PaddleClas/docs/zh_CN/models/DLA.md b/src/PaddleClas/docs/zh_CN/models/DLA.md new file mode 100644 index 0000000..3612b9e --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/DLA.md @@ -0,0 +1,45 @@ +# DLA 系列 +---- +## 目录 +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) + + + +## 1. 概述 + +DLA(Deep Layer Aggregation)。 视觉识别需要丰富的表示形式，其范围从低到高，范围从小到大，分辨率从精细到粗糙。即使卷积网络中的要素深度很深，仅靠隔离层还是不够的：将这些表示法进行复合和聚合可改善对内容和位置的推断。尽管已合并了残差连接以组合各层，但是这些连接本身是“浅”的，并且只能通过简单的一步操作来融合。作者通过更深层的聚合来增强标准体系结构，以更好地融合各层的信息。Deep Layer Aggregation 结构迭代地和分层地合并了特征层次结构，以使网络具有更高的准确性和更少的参数。跨体系结构和任务的实验表明，与现有的分支和合并方案相比，Deep Layer Aggregation 可提高识别和分辨率。[论文地址](https://arxiv.org/abs/1707.06484)。 + + + +## 2. 精度、FLOPS 和参数量 + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:-----------------:|:----------:|:---------:|:---------:|:---------:| +| DLA34 | 15.8 | 3.1 | 76.03 | 92.98 | +| DLA46_c | 1.3 | 0.5 | 63.21 | 85.30 | +| DLA46x_c | 1.1 | 0.5 | 64.36 | 86.01 | +| DLA60 | 22.0 | 4.2 | 76.10 | 92.92 | +| DLA60x | 17.4 | 3.5 | 77.53 | 93.78 | +| DLA60x_c | 1.3 | 0.6 | 66.45 | 87.54 | +| DLA102 | 33.3 | 7.2 | 78.93 | 94.52 | +| DLA102x | 26.4 | 5.9 | 78.10 | 94.00 | +| DLA102x2 | 41.4 | 9.3 | 78.85 | 94.45 | +| DLA169 | 53.5 | 11.6 | 78.09 | 94.09 | + + + +## 3. 基于 V100 GPU 的预测速度 + +| 模型 | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| -------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| DLA102 | 224 | 256 | 4.95 | 8.08 | 12.40 | +| DLA102x2 | 224 | 256 | 19.58 | 23.97 | 31.37 | +| DLA102x | 224 | 256 | 11.12 | 15.60 | 20.37 | +| DLA169 | 224 | 256 | 7.70 | 12.25 | 18.90 | +| DLA34 | 224 | 256 | 1.83 | 3.37 | 5.98 | +| DLA46_c | 224 | 256 | 1.06 | 2.08 | 3.23 | +| DLA60 | 224 | 256 | 2.78 | 5.36 | 8.29 | +| DLA60x_c | 224 | 256 | 1.79 | 3.68 | 5.19 | +| DLA60x | 224 | 256 | 5.98 | 9.24 | 12.52 | \ No newline at end of file diff --git a/src/PaddleClas/docs/zh_CN/models/DPN_DenseNet.md b/src/PaddleClas/docs/zh_CN/models/DPN_DenseNet.md new file mode 100644 index 0000000..3a8a002 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/DPN_DenseNet.md @@ -0,0 +1,79 @@ +# DPN 与 DenseNet 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + + +## 1. 概述 +DenseNet 是 2017 年 CVPR best paper 提出的一种新的网络结构，该网络设计了一种新的跨层连接的 block，即 dense-block。相比 ResNet 中的 bottleneck，dense-block 设计了一个更激进的密集连接机制，即互相连接所有的层，每个层都会接受其前面所有层作为其额外的输入。DenseNet 将所有的 dense-block 堆叠，组合成了一个密集连接型网络。密集的连接方式使得 DenseNe 更容易进行梯度的反向传播，使得网络更容易训练。 +DPN 的全称是 Dual Path Networks，即双通道网络。该网络是由 DenseNet 和 ResNeXt 结合的一个网络，其证明了 DenseNet 能从靠前的层级中提取到新的特征，而 ResNeXt 本质上是对之前层级中已提取特征的复用。作者进一步分析发现，ResNeXt 对特征有高复用率，但冗余度低，DenseNet 能创造新特征，但冗余度高。结合二者结构的优势，作者设计了 DPN 网络。最终 DPN 网络在同样 FLOPS 和参数量下，取得了比 ResNeXt 与 DenseNet 更好的结果。 + +该系列模型的 FLOPS、参数量以及 T4 GPU 上的预测耗时如下图所示。 + +![](../../images/models/T4_benchmark/t4.fp32.bs4.DPN.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.DPN.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.DPN.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.DPN.png) + +目前 PaddleClas 开源的这两类模型的预训练模型一共有 10 个，其指标如上图所示，可以看到，在相同的 FLOPS 和参数量下，相比 DenseNet，DPN 拥有更高的精度。但是由于 DPN 有更多的分支，所以其推理速度要慢于 DenseNet。由于 DenseNet264 的网络层数最深，所以该网络是 DenseNet 系列模型中参数量最大的网络，DenseNet161 的网络的宽度最大，导致其是该系列中网络中计算量最大、精度最高的网络。从推理速度来看，计算量大且精度高的的 DenseNet161 比 DenseNet264 具有更快的速度，所以其比 DenseNet264 具有更大的优势。 + +对于 DPN 系列网络，模型的 FLOPS 和参数量越大，模型的精度越高。其中，由于 DPN107 的网络宽度最大，所以其是该系列网络中参数量与计算量最大的网络。 + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| DenseNet121 | 0.757 | 0.926 | 0.750 | | 5.690 | 7.980 | +| DenseNet161 | 0.786 | 0.941 | 0.778 | | 15.490 | 28.680 | +| DenseNet169 | 0.768 | 0.933 | 0.764 | | 6.740 | 14.150 | +| DenseNet201 | 0.776 | 0.937 | 0.775 | | 8.610 | 20.010 | +| DenseNet264 | 0.780 | 0.939 | 0.779 | | 11.540 | 33.370 | +| DPN68 | 0.768 | 0.934 | 0.764 | 0.931 | 4.030 | 10.780 | +| DPN92 | 0.799 | 0.948 | 0.793 | 0.946 | 12.540 | 36.290 | +| DPN98 | 0.806 | 0.951 | 0.799 | 0.949 | 22.220 | 58.460 | +| DPN107 | 0.809 | 0.953 | 0.802 | 0.951 | 35.060 | 82.970 | +| DPN131 | 0.807 | 0.951 | 0.801 | 0.949 | 30.510 | 75.360 | + + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------|-----------|-------------------|-------------------|-------------------|-------------------| +| DenseNet121 | 224 | 256 | 3.40 | 6.94 | 9.17 | +| DenseNet161 | 224 | 256 | 7.06 | 14.37 | 19.55 | +| DenseNet169 | 224 | 256 | 5.00 | 10.29 | 12.84 | +| DenseNet201 | 224 | 256 | 6.38 | 13.72 | 17.17 | +| DenseNet264 | 224 | 256 | 9.34 | 20.95 | 25.41 | +| DPN68 | 224 | 256 | 8.18 | 11.40 | 14.82 | +| DPN92 | 224 | 256 | 12.48 | 20.04 | 25.10 | +| DPN98 | 224 | 256 | 14.70 | 25.55 | 35.12 | +| DPN107 | 224 | 256 | 19.46 | 35.62 | 50.22 | +| DPN131 | 224 | 256 | 19.64 | 34.60 | 47.42 | + + + +## 4. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| DenseNet121 | 224 | 256 | 4.16436 | 7.2126 | 10.50221 | 4.40447 | 9.32623 | 15.25175 | +| DenseNet161 | 224 | 256 | 9.27249 | 14.25326 | 20.19849 | 10.39152 | 22.15555 | 35.78443 | +| DenseNet169 | 224 | 256 | 6.11395 | 10.28747 | 13.68717 | 6.43598 | 12.98832 | 20.41964 | +| DenseNet201 | 224 | 256 | 7.9617 | 13.4171 | 17.41949 | 8.20652 | 17.45838 | 27.06309 | +| DenseNet264 | 224 | 256 | 11.70074 | 19.69375 | 24.79545 | 12.14722 | 26.27707 | 40.01905 | +| DPN68 | 224 | 256 | 11.7827 | 13.12652 | 16.19213 | 11.64915 | 12.82807 | 18.57113 | +| DPN92 | 224 | 256 | 18.56026 | 20.35983 | 29.89544 | 18.15746 | 23.87545 | 38.68821 | +| DPN98 | 224 | 256 | 21.70508 | 24.7755 | 40.93595 | 21.18196 | 33.23925 | 62.77751 | +| DPN107 | 224 | 256 | 27.84462 | 34.83217 | 60.67903 | 27.62046 | 52.65353 | 100.11721 | +| DPN131 | 224 | 256 | 28.58941 | 33.01078 | 55.65146 | 28.33119 | 46.19439 | 89.24904 | diff --git a/src/PaddleClas/docs/zh_CN/models/ESNet.md b/src/PaddleClas/docs/zh_CN/models/ESNet.md new file mode 100644 index 0000000..cff1354 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/ESNet.md @@ -0,0 +1,25 @@ +# ESNet 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPs 和参数量](#2) + + + +## 1. 概述 + +ESNet(Enhanced ShuffleNet)是百度自研的一个轻量级网络，该网络在 ShuffleNetV2 的基础上融合了 MobileNetV3、GhostNet、PPLCNet 的优点，组合成了一个在 ARM 设备上速度更快、精度更高的网络，由于其出色的表现，所以在 PaddleDetection 推出的 [PP-PicoDet](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/picodet) 使用了该模型做 backbone，配合更强的目标检测算法，最终的指标一举刷新了目标检测模型在 ARM 设备上的 SOTA 指标。 + + + +## 2.精度、FLOPs 和参数量 + +| Models | Top1 | Top5 | FLOPs
(M) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:| +| ESNet_x0_25 | 62.48 | 83.46 | 30.9 | 2.83 | +| ESNet_x0_5 | 68.82 | 88.04 | 67.3 | 3.25 | +| ESNet_x0_75 | 72.24 | 90.45 | 123.7 | 3.87 | +| ESNet_x1_0 | 73.92 | 91.40 | 197.3 | 4.64 | + +关于 Inference speed 等信息，敬请期待。 diff --git a/src/PaddleClas/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md b/src/PaddleClas/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md new file mode 100644 index 0000000..dfe68ac --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md @@ -0,0 +1,90 @@ +# EfficientNet 与 ResNeXt101_wsl 系列 +----- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + +## 1. 概述 +EfficientNet 是 Google 于 2019 年发布的一个基于 NAS 的轻量级网络，其中 EfficientNetB7 刷新了当时 ImageNet-1k 的分类准确率。在该文章中，作者指出，传统的提升神经网络性能的方法主要是从网络的宽度、网络的深度、以及输入图片的分辨率入手，但是作者通过实验发现，平衡这三个维度对精度和效率的提升至关重要，于是，作者通过一系列的实验中总结出了如何同时平衡这三个维度的放缩，与此同时，基于这种放缩方法，作者在 EfficientNet_B0 的基础上，构建了 EfficientNet 系列中 B1-B7 共 7 个网络，并在同样 FLOPS 与参数量的情况下，精度达到了 state-of-the-art 的效果。 + +ResNeXt 是 facebook 于 2016 年提出的一种对 ResNet 的改进版网络。在 2019 年，facebook 通过弱监督学习研究了该系列网络在 ImageNet 上的精度上限，为了区别之前的 ResNeXt 网络，该系列网络的后缀为 wsl，其中 wsl 是弱监督学习（weakly-supervised-learning）的简称。为了能有更强的特征提取能力，研究者将其网络宽度进一步放大，其中最大的 ResNeXt101_32x48d_wsl 拥有 8 亿个参数，将其在 9.4 亿的弱标签图片下训练并在 ImageNet-1k 上做 finetune，最终在 ImageNet-1k 的 top-1 达到了 85.4%，这也是迄今为止在 ImageNet-1k 的数据集上以 224x224 的分辨率下精度最高的网络。Fix-ResNeXt 中，作者使用了更大的图像分辨率，针对训练图片和验证图片数据预处理不一致的情况下做了专门的 Fix 策略，并使得 ResNeXt101_32x48d_wsl 拥有了更高的精度，由于其用到了 Fix 策略，故命名为 Fix-ResNeXt101_32x48d_wsl。 + + +该系列模型的 FLOPS、参数量以及 T4 GPU 上的预测耗时如下图所示。 + +![](../../images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.EfficientNet.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs1.EfficientNet.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs1.EfficientNet.png) + +目前 PaddleClas 开源的这两类模型的预训练模型一共有 14 个。从上图中可以看出 EfficientNet 系列网络优势非常明显，ResNeXt101_wsl 系列模型由于用到了更多的数据，最终的精度也更高。EfficientNet_B0_Small 是去掉了 SE_block 的 EfficientNet_B0，其具有更快的推理速度。 + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ResNeXt101_
32x8d_wsl | 0.826 | 0.967 | 0.822 | 0.964 | 29.140 | 78.440 | +| ResNeXt101_
32x16d_wsl | 0.842 | 0.973 | 0.842 | 0.972 | 57.550 | 152.660 | +| ResNeXt101_
32x32d_wsl | 0.850 | 0.976 | 0.851 | 0.975 | 115.170 | 303.110 | +| ResNeXt101_
32x48d_wsl | 0.854 | 0.977 | 0.854 | 0.976 | 173.580 | 456.200 | +| Fix_ResNeXt101_
32x48d_wsl | 0.863 | 0.980 | 0.864 | 0.980 | 354.230 | 456.200 | +| EfficientNetB0 | 0.774 | 0.933 | 0.773 | 0.935 | 0.720 | 5.100 | +| EfficientNetB1 | 0.792 | 0.944 | 0.792 | 0.945 | 1.270 | 7.520 | +| EfficientNetB2 | 0.799 | 0.947 | 0.803 | 0.950 | 1.850 | 8.810 | +| EfficientNetB3 | 0.812 | 0.954 | 0.817 | 0.956 | 3.430 | 11.840 | +| EfficientNetB4 | 0.829 | 0.962 | 0.830 | 0.963 | 8.290 | 18.760 | +| EfficientNetB5 | 0.836 | 0.967 | 0.837 | 0.967 | 19.510 | 29.610 | +| EfficientNetB6 | 0.840 | 0.969 | 0.842 | 0.968 | 36.270 | 42.000 | +| EfficientNetB7 | 0.843 | 0.969 | 0.844 | 0.971 | 72.350 | 64.920 | +| EfficientNetB0_
small | 0.758 | 0.926 | | | 0.720 | 4.650 | + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------------------------|-----------|-------------------|-------------------------------|-------------------------------|-------------------------------| +| ResNeXt101_
32x8d_wsl | 224 | 256 | 13.55 | 23.39 | 36.18 | +| ResNeXt101_
32x16d_wsl | 224 | 256 | 21.96 | 38.35 | 63.29 | +| ResNeXt101_
32x32d_wsl | 224 | 256 | 37.28 | 76.50 | 121.56 | +| ResNeXt101_
32x48d_wsl | 224 | 256 | 55.07 | 124.39 | 205.01 | +| Fix_ResNeXt101_
32x48d_wsl | 320 | 320 | 55.01 | 122.63 | 204.66 | +| EfficientNetB0 | 224 | 256 | 1.96 | 3.71 | 5.56 | +| EfficientNetB1 | 240 | 272 | 2.88 | 5.40 | 7.63 | +| EfficientNetB2 | 260 | 292 | 3.26 | 6.20 | 9.17 | +| EfficientNetB3 | 300 | 332 | 4.52 | 8.85 | 13.54 | +| EfficientNetB4 | 380 | 412 | 6.78 | 15.47 | 24.95 | +| EfficientNetB5 | 456 | 488 | 10.97 | 27.24 | 45.93 | +| EfficientNetB6 | 528 | 560 | 17.09 | 43.32 | 76.90 | +| EfficientNetB7 | 600 | 632 | 25.91 | 71.23 | 128.20 | +| EfficientNetB0_
small | 224 | 256 | 1.24 | 2.59 | 3.92 | + + + + +## 4. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|---------------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| ResNeXt101_
32x8d_wsl | 224 | 256 | 18.19374 | 21.93529 | 34.67802 | 18.52528 | 34.25319 | 67.2283 | +| ResNeXt101_
32x16d_wsl | 224 | 256 | 18.52609 | 36.8288 | 62.79947 | 25.60395 | 71.88384 | 137.62327 | +| ResNeXt101_
32x32d_wsl | 224 | 256 | 33.51391 | 70.09682 | 125.81884 | 54.87396 | 160.04337 | 316.17718 | +| ResNeXt101_
32x48d_wsl | 224 | 256 | 50.97681 | 137.60926 | 190.82628 | 99.01698256 | 315.91261 | 551.83695 | +| Fix_ResNeXt101_
32x48d_wsl | 320 | 320 | 78.62869 | 191.76039 | 317.15436 | 160.0838242 | 595.99296 | 1151.47384 | +| EfficientNetB0 | 224 | 256 | 3.40122 | 5.95851 | 9.10801 | 3.442 | 6.11476 | 9.3304 | +| EfficientNetB1 | 240 | 272 | 5.25172 | 9.10233 | 14.11319 | 5.3322 | 9.41795 | 14.60388 | +| EfficientNetB2 | 260 | 292 | 5.91052 | 10.5898 | 17.38106 | 6.29351 | 10.95702 | 17.75308 | +| EfficientNetB3 | 300 | 332 | 7.69582 | 16.02548 | 27.4447 | 7.67749 | 16.53288 | 28.5939 | +| EfficientNetB4 | 380 | 412 | 11.55585 | 29.44261 | 53.97363 | 12.15894 | 30.94567 | 57.38511 | +| EfficientNetB5 | 456 | 488 | 19.63083 | 56.52299 | - | 20.48571 | 61.60252 | - | +| EfficientNetB6 | 528 | 560 | 30.05911 | - | - | 32.62402 | - | - | +| EfficientNetB7 | 600 | 632 | 47.86087 | - | - | 53.93823 | - | - | +| EfficientNetB0_small | 224 | 256 | 2.39166 | 4.36748 | 6.96002 | 2.3076 | 4.71886 | 7.21888 | diff --git a/src/PaddleClas/docs/zh_CN/models/HRNet.md b/src/PaddleClas/docs/zh_CN/models/HRNet.md new file mode 100644 index 0000000..179c946 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/HRNet.md @@ -0,0 +1,73 @@ +# HRNet 系列 +----- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + + +## 1. 概述 +HRNet 是 2019 年由微软亚洲研究院提出的一种全新的神经网络，不同于以往的卷积神经网络，该网络在网络深层仍然可以保持高分辨率，因此预测的关键点热图更准确，在空间上也更精确。此外，该网络在对分辨率敏感的其他视觉任务中，如检测、分割等，表现尤为优异。 + +该系列模型的 FLOPS、参数量以及 T4 GPU 上的预测耗时如下图所示。 + +![](../../images/models/T4_benchmark/t4.fp32.bs4.HRNet.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.HRNet.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.HRNet.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.HRNet.png) + +目前 PaddleClas 开源的这类模型的预训练模型一共有 7 个，其指标如图所示，其中 HRNet_W48_C 指标精度异常的原因可能是因为网络训练的正常波动。 + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| HRNet_W18_C | 0.769 | 0.934 | 0.768 | 0.934 | 4.140 | 21.290 | +| HRNet_W18_C_ssld | 0.816 | 0.958 | 0.768 | 0.934 | 4.140 | 21.290 | +| HRNet_W30_C | 0.780 | 0.940 | 0.782 | 0.942 | 16.230 | 37.710 | +| HRNet_W32_C | 0.783 | 0.942 | 0.785 | 0.942 | 17.860 | 41.230 | +| HRNet_W40_C | 0.788 | 0.945 | 0.789 | 0.945 | 25.410 | 57.550 | +| HRNet_W44_C | 0.790 | 0.945 | 0.789 | 0.944 | 29.790 | 67.060 | +| HRNet_W48_C | 0.790 | 0.944 | 0.793 | 0.945 | 34.580 | 77.470 | +| HRNet_W48_C_ssld | 0.836 | 0.968 | 0.793 | 0.945 | 34.580 | 77.470 | +| HRNet_W64_C | 0.793 | 0.946 | 0.795 | 0.946 | 57.830 | 128.060 | +| SE_HRNet_W64_C_ssld | 0.847 | 0.973 | | | 57.830 | 128.970 | + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------|-----------|-------------------|-------------------|-------------------|-------------------| +| HRNet_W18_C | 224 | 256 | 6.66 | 8.94 | 11.95 | +| HRNet_W18_C_ssld | 224 | 256 | 6.66 | 8.92 | 11.93 | +| HRNet_W30_C | 224 | 256 | 8.61 | 11.40 | 15.23 | +| HRNet_W32_C | 224 | 256 | 8.54 | 11.58 | 15.57 | +| HRNet_W40_C | 224 | 256 | 9.83 | 15.02 | 20.92 | +| HRNet_W44_C | 224 | 256 | 10.62 | 16.18 | 25.92 | +| HRNet_W48_C | 224 | 256 | 11.07 | 17.06 | 27.28 | +| HRNet_W48_C_ssld | 224 | 256 | 11.09 | 17.04 | 27.28 | +| HRNet_W64_C | 224 | 256 | 13.82 | 21.15 | 35.51 | + + + +## 4. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| HRNet_W18_C | 224 | 256 | 6.79093 | 11.50986 | 17.67244 | 7.40636 | 13.29752 | 23.33445 | +| HRNet_W18_C_ssld | 224 | 256 | 6.79093 | 11.50986 | 17.67244 | 7.40636 | 13.29752 | 23.33445 | +| HRNet_W30_C | 224 | 256 | 8.98077 | 14.08082 | 21.23527 | 9.57594 | 17.35485 | 32.6933 | +| HRNet_W32_C | 224 | 256 | 8.82415 | 14.21462 | 21.19804 | 9.49807 | 17.72921 | 32.96305 | +| HRNet_W40_C | 224 | 256 | 11.4229 | 19.1595 | 30.47984 | 12.12202 | 25.68184 | 48.90623 | +| HRNet_W44_C | 224 | 256 | 12.25778 | 22.75456 | 32.61275 | 13.19858 | 32.25202 | 59.09871 | +| HRNet_W48_C | 224 | 256 | 12.65015 | 23.12886 | 33.37859 | 13.70761 | 34.43572 | 63.01219 | +| HRNet_W48_C_ssld | 224 | 256 | 12.65015 | 23.12886 | 33.37859 | 13.70761 | 34.43572 | 63.01219 | +| HRNet_W64_C | 224 | 256 | 15.10428 | 27.68901 | 40.4198 | 17.57527 | 47.9533 | 97.11228 | +| SE_HRNet_W64_C_ssld | 224 | 256 | 32.33651 | 69.31189 | 116.07245 | 31.69770 | 94.99546 | 174.45766 | diff --git a/src/PaddleClas/docs/zh_CN/models/HarDNet.md b/src/PaddleClas/docs/zh_CN/models/HarDNet.md new file mode 100644 index 0000000..3f75fad --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/HarDNet.md @@ -0,0 +1,35 @@ +# HarDNet 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) + + +## 1. 概述 + +HarDNet（Harmonic DenseNet）是 2019 年由国立清华大学提出的一种全新的神经网络，在低 MAC 和内存流量的条件下实现了高效率。与 FC-DenseNet-103，DenseNet-264，ResNet-50，ResNet-152 和 SSD-VGG 相比，新网络的推理时间减少了 35%，36%，30%，32% 和 45%。我们使用了包括 Nvidia Profiler 和 ARM Scale-Sim 在内的工具来测量内存流量，并验证推理延迟确实与内存流量消耗成正比，并且所提议的网络消耗的内存流量很低。[论文地址](https://arxiv.org/abs/1909.00948)。 + + + +## 2. 精度、FLOPS 和参数量 + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:---------------------:|:----------:|:---------:|:---------:|:---------:| +| HarDNet68 | 17.6 | 4.3 | 75.46 | 92.65 | +| HarDNet85 | 36.7 | 9.1 | 77.44 | 93.55 | +| HarDNet39_ds | 3.5 | 0.4 | 71.33 | 89.98 | +| HarDNet68_ds | 4.2 | 0.8 | 73.62 | 91.52 | + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| ------------ | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| HarDNet68 | 224 | 256 | 3.58 | 8.53 | 11.58 | +| HarDNet85 | 224 | 256 | 6.24 | 14.85 | 20.57 | +| HarDNet39_ds | 224 | 256 | 1.40 | 2.30 | 3.33 | +| HarDNet68_ds | 224 | 256 | 2.26 | 3.34 | 5.06 | + diff --git a/src/PaddleClas/docs/zh_CN/models/Inception.md b/src/PaddleClas/docs/zh_CN/models/Inception.md new file mode 100644 index 0000000..bed40b3 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/Inception.md @@ -0,0 +1,80 @@ +# Inception 系列 +----- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + + +## 1. 概述 + +GoogLeNet 是 2014 年由 Google 设计的一种新的神经网络结构，其与 VGG 网络并列成为当年 ImageNet 挑战赛的双雄。GoogLeNet 首次引入 Inception 结构，在网络中堆叠该结构使得网络层数达到了 22 层，这也是卷积网络首次超过 20 层的标志。由于在 Inception 结构中使用了 1x1 的卷积用于通道数降维，并且使用了 Global-pooling 代替传统的多 fc 层加工特征的方式，最终的 GoogLeNet 网络的 FLOPS 和参数量远小于 VGG 网络，成为当时神经网络设计的一道亮丽风景线。 + +InceptionV3 是 Google 对 InceptionV2 的一种改进。首先，InceptionV3 对 Inception 模块进行了优化，同时设计和使用了更多种类的 Inception 模块，与此同时，InceptionV3 中的部分 Inception 模块将较大的方形二维卷积拆成两个较小的非对称卷积，这样可以大幅度节省参数量。 + +Xception 是 Google 继 Inception 后提出的对 InceptionV3 的另一种改进。在 Xception 中，作者使用了深度可分离卷积代替了传统的卷积操作，该操作大大节省了网络的 FLOPS 和参数量，但是精度反而有所提升。在 DeeplabV3+ 中，作者将 Xception 做了进一步的改进，同时增加了 Xception 的层数，设计出了 Xception65 和 Xception71 的网络。 + +InceptionV4 是 2016 年由 Google 设计的新的神经网络，当时残差结构风靡一时，但是作者认为仅使用 Inception 结构也可以达到很高的性能。InceptionV4 使用了更多的 Inception module，在 ImageNet 上的精度再创新高。 + + +该系列模型的 FLOPS、参数量以及 T4 GPU 上的预测耗时如下图所示。 + +![](../../images/models/T4_benchmark/t4.fp32.bs4.Inception.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.Inception.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.Inception.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.Inception.png) + +上图反映了 Xception 系列和 InceptionV4 的精度和其他指标的关系。其中 Xception_deeplab 与论文结构保持一致，Xception 是 PaddleClas 的改进模型，在预测速度基本不变的情况下，精度提升约 0.6%。关于该改进模型的详细介绍正在持续更新中，敬请期待。 + + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| GoogLeNet | 0.707 | 0.897 | 0.698 | | 2.880 | 8.460 | +| Xception41 | 0.793 | 0.945 | 0.790 | 0.945 | 16.740 | 22.690 | +| Xception41
_deeplab | 0.796 | 0.944 | | | 18.160 | 26.730 | +| Xception65 | 0.810 | 0.955 | | | 25.950 | 35.480 | +| Xception65
_deeplab | 0.803 | 0.945 | | | 27.370 | 39.520 | +| Xception71 | 0.811 | 0.955 | | | 31.770 | 37.280 | +| InceptionV3 | 0.791 | 0.946 | 0.788 | 0.944 | 11.460 | 23.830 | +| InceptionV4 | 0.808 | 0.953 | 0.800 | 0.950 | 24.570 | 42.680 | + + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|------------------------|-----------|-------------------|------------------------|------------------------|------------------------| +| GoogLeNet | 224 | 256 | 1.41 | 3.25 | 5.00 | +| Xception41 | 299 | 320 | 3.58 | 8.76 | 16.61 | +| Xception41_
deeplab | 299 | 320 | 3.81 | 9.16 | 17.20 | +| Xception65 | 299 | 320 | 5.45 | 12.78 | 24.53 | +| Xception65_
deeplab | 299 | 320 | 5.65 | 13.08 | 24.61 | +| Xception71 | 299 | 320 | 6.19 | 15.34 | 29.21 | +| InceptionV4 | 299 | 320 | 8.93 | 15.17 | 21.56 | + + + + +## 4. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|--------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| GoogLeNet | 299 | 320 | 1.75451 | 3.39931 | 4.71909 | 1.88038 | 4.48882 | 6.94035 | +| Xception41 | 299 | 320 | 2.91192 | 7.86878 | 15.53685 | 4.96939 | 17.01361 | 32.67831 | +| Xception41_
deeplab | 299 | 320 | 2.85934 | 7.2075 | 14.01406 | 5.33541 | 17.55938 | 33.76232 | +| Xception65 | 299 | 320 | 4.30126 | 11.58371 | 23.22213 | 7.26158 | 25.88778 | 53.45426 | +| Xception65_
deeplab | 299 | 320 | 4.06803 | 9.72694 | 19.477 | 7.60208 | 26.03699 | 54.74724 | +| Xception71 | 299 | 320 | 4.80889 | 13.5624 | 27.18822 | 8.72457 | 31.55549 | 69.31018 | +| InceptionV3 | 299 | 320 | 3.67502 | 6.36071 | 9.82645 | 6.64054 | 13.53630 | 22.17355 | +| InceptionV4 | 299 | 320 | 9.50821 | 13.72104 | 20.27447 | 12.99342 | 25.23416 | 43.56121 | diff --git a/src/PaddleClas/docs/zh_CN/models/LeViT.md b/src/PaddleClas/docs/zh_CN/models/LeViT.md new file mode 100644 index 0000000..5f0e480 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/LeViT.md @@ -0,0 +1,27 @@ +# LeViT + +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) + + + +## 1. 概述 +LeViT 是一种快速推理的、用于图像分类任务的混合神经网络。其设计之初考虑了网络模型在不同的硬件平台上的性能，因此能够更好地反映普遍应用的真实场景。通过大量实验，作者找到了卷积神经网络与 Transformer 体系更好的结合方式，并且提出了 attention-based 方法，用于整合 Transformer 中的位置信息编码。[论文地址](https://arxiv.org/abs/2104.01136)。 + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(M) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| LeViT-128S | 0.7598 | 0.9269 | 0.766 | 0.929 | 305 | 7.8 | +| LeViT-128 | 0.7810 | 0.9371 | 0.786 | 0.940 | 406 | 9.2 | +| LeViT-192 | 0.7934 | 0.9446 | 0.800 | 0.947 | 658 | 11 | +| LeViT-256 | 0.8085 | 0.9497 | 0.816 | 0.954 | 1120 | 19 | +| LeViT-384 | 0.8191 | 0.9551 | 0.826 | 0.960 | 2353 | 39 | + + +**注**：与 Reference 的精度差异源于数据预处理不同及未使用蒸馏的 head 作为输出。 diff --git a/src/PaddleClas/docs/zh_CN/models/MixNet.md b/src/PaddleClas/docs/zh_CN/models/MixNet.md new file mode 100644 index 0000000..eaf45d3 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/MixNet.md @@ -0,0 +1,40 @@ +# MixNet 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) + + + +## 1. 概述 + +MixNet 是谷歌出的一篇关于轻量级网络的文章，主要工作就在于探索不同大小的卷积核的组合。作者发现目前网络有以下两个问题： + +- 小的卷积核感受野小，参数少，但是准确率不高 +- 大的卷积核感受野大，准确率相对略高，但是参数也相对增加了很多 + +为了解决上面两个问题，文中提出一种新的混合深度分离卷积(MDConv)(mixed depthwise convolution)，将不同的核大小混合在一个卷积运算中，并且基于 AutoML 的搜索空间，提出了一系列的网络叫做 MixNets，在 ImageNet 上取得了较好的效果。[论文地址](https://arxiv.org/pdf/1907.09595.pdf) + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1| FLOPS
(M) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|----| +| MixNet_S | 76.28 | 92.99 | 75.8 | 252.977 | 4.167 | +| MixNet_M | 77.67 | 93.64 | 77.0 | 357.119 | 5.065 | +| MixNet_L | 78.60 | 94.37 | 78.9 | 579.017 | 7.384 | + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| -------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| MixNet_S | 224 | 256 | 2.31 | 3.63 | 5.20 | +| MixNet_M | 224 | 256 | 2.84 | 4.60 | 6.62 | +| MixNet_L | 224 | 256 | 3.16 | 5.55 | 8.03 | + +关于 Inference speed 等信息，敬请期待。 diff --git a/src/PaddleClas/docs/zh_CN/models/Mobile.md b/src/PaddleClas/docs/zh_CN/models/Mobile.md new file mode 100644 index 0000000..c4cede5 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/Mobile.md @@ -0,0 +1,207 @@ +# 移动端系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 SD855 的预测速度和存储大小](#3) +* [4. 基于 V100 GPU 的预测速度](#4) +* [5. 基于 T4 GPU 的预测速度](#5) + + + +## 1. 概述 +MobileNetV1 是 Google 于 2017 年发布的用于移动设备或嵌入式设备中的网络。该网络将传统的卷积操作替换深度可分离卷积，即 Depthwise 卷积和 Pointwise 卷积的组合，相比传统的卷积操作，该组合可以大大节省参数量和计算量。与此同时，MobileNetV1 也可以用于目标检测、图像分割等其他视觉任务中。 + +MobileNetV2 是 Google 继 MobileNetV1 提出的一种轻量级网络。相比 MobileNetV1，MobileNetV2 提出了 Linear bottlenecks 与 Inverted residual block 作为网络基本结构，通过大量地堆叠这些基本模块，构成了 MobileNetV2 的网络结构。最终，在 FLOPS 只有 MobileNetV1 的一半的情况下取得了更高的分类精度。 + +ShuffleNet 系列网络是旷视提出的轻量化网络结构，到目前为止，该系列网络一共有两种典型的结构，即 ShuffleNetV1 与 ShuffleNetV2。ShuffleNet 中的 Channel Shuffle 操作可以将组间的信息进行交换，并且可以实现端到端的训练。在 ShuffleNetV2 的论文中，作者提出了设计轻量级网络的四大准则，并且根据四大准则与 ShuffleNetV1 的不足，设计了 ShuffleNetV2 网络。 + +MobileNetV3 是 Google 于 2019 年提出的一种基于 NAS 的新的轻量级网络，为了进一步提升效果，将 relu 和 sigmoid 激活函数分别替换为 hard_swish 与 hard_sigmoid 激活函数，同时引入了一些专门减小网络计算量的改进策略。 + +GhostNet 是华为于 2020 年提出的一种全新的轻量化网络结构，通过引入 ghost module，大大减缓了传统深度网络中特征的冗余计算问题，使得网络的参数量和计算量大大降低。 + +![](../../images/models/mobile_arm_top1.png) + +![](../../images/models/mobile_arm_storage.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.mobile_trt.params.png) + + +目前 PaddleClas 开源的的移动端系列的预训练模型一共有 35 个，其指标如图所示。从图片可以看出，越新的轻量级模型往往有更优的表现，MobileNetV3 代表了目前主流的轻量级神经网络结构。在 MobileNetV3 中，作者为了获得更高的精度，在 global-avg-pooling 后使用了 1x1 的卷积。该操作大幅提升了参数量但对计算量影响不大，所以如果从存储角度评价模型的优异程度，MobileNetV3 优势不是很大，但由于其更小的计算量，使得其有更快的推理速度。此外，我们模型库中的 ssld 蒸馏模型表现优异，从各个考量角度下，都刷新了当前轻量级模型的精度。由于 MobileNetV3 模型结构复杂，分支较多，对 GPU 并不友好，GPU 预测速度不如 MobileNetV1。GhostNet 于 2020 年提出，通过引入 ghost 的网络设计理念，大大降低了计算量和参数量，同时在精度上也超过前期最高的 MobileNetV3 网络结构。 + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| MobileNetV1_x0_25 | 0.514 | 0.755 | 0.506 | | 0.070 | 0.460 | +| MobileNetV1_x0_5 | 0.635 | 0.847 | 0.637 | | 0.280 | 1.310 | +| MobileNetV1_x0_75 | 0.688 | 0.882 | 0.684 | | 0.630 | 2.550 | +| MobileNetV1 | 0.710 | 0.897 | 0.706 | | 1.110 | 4.190 | +| MobileNetV1_ssld | 0.779 | 0.939 | | | 1.110 | 4.190 | +| MobileNetV2_x0_25 | 0.532 | 0.765 | | | 0.050 | 1.500 | +| MobileNetV2_x0_5 | 0.650 | 0.857 | 0.654 | 0.864 | 0.170 | 1.930 | +| MobileNetV2_x0_75 | 0.698 | 0.890 | 0.698 | 0.896 | 0.350 | 2.580 | +| MobileNetV2 | 0.722 | 0.907 | 0.718 | 0.910 | 0.600 | 3.440 | +| MobileNetV2_x1_5 | 0.741 | 0.917 | | | 1.320 | 6.760 | +| MobileNetV2_x2_0 | 0.752 | 0.926 | | | 2.320 | 11.130 | +| MobileNetV2_ssld | 0.7674 | 0.9339 | | | 0.600 | 3.440 | +| MobileNetV3_large_
x1_25 | 0.764 | 0.930 | 0.766 | | 0.714 | 7.440 | +| MobileNetV3_large_
x1_0 | 0.753 | 0.923 | 0.752 | | 0.450 | 5.470 | +| MobileNetV3_large_
x0_75 | 0.731 | 0.911 | 0.733 | | 0.296 | 3.910 | +| MobileNetV3_large_
x0_5 | 0.692 | 0.885 | 0.688 | | 0.138 | 2.670 | +| MobileNetV3_large_
x0_35 | 0.643 | 0.855 | 0.642 | | 0.077 | 2.100 | +| MobileNetV3_small_
x1_25 | 0.707 | 0.895 | 0.704 | | 0.195 | 3.620 | +| MobileNetV3_small_
x1_0 | 0.682 | 0.881 | 0.675 | | 0.123 | 2.940 | +| MobileNetV3_small_
x0_75 | 0.660 | 0.863 | 0.654 | | 0.088 | 2.370 | +| MobileNetV3_small_
x0_5 | 0.592 | 0.815 | 0.580 | | 0.043 | 1.900 | +| MobileNetV3_small_
x0_35 | 0.530 | 0.764 | 0.498 | | 0.026 | 1.660 | +| MobileNetV3_small_
x0_35_ssld | 0.556 | 0.777 | 0.498 | | 0.026 | 1.660 | +| MobileNetV3_large_
x1_0_ssld | 0.790 | 0.945 | | | 0.450 | 5.470 | +| MobileNetV3_large_
x1_0_ssld_int8 | 0.761 | | | | | | +| MobileNetV3_small_
x1_0_ssld | 0.713 | 0.901 | | | 0.123 | 2.940 | +| ShuffleNetV2 | 0.688 | 0.885 | 0.694 | | 0.280 | 2.260 | +| ShuffleNetV2_x0_25 | 0.499 | 0.738 | | | 0.030 | 0.600 | +| ShuffleNetV2_x0_33 | 0.537 | 0.771 | | | 0.040 | 0.640 | +| ShuffleNetV2_x0_5 | 0.603 | 0.823 | 0.603 | | 0.080 | 1.360 | +| ShuffleNetV2_x1_5 | 0.716 | 0.902 | 0.726 | | 0.580 | 3.470 | +| ShuffleNetV2_x2_0 | 0.732 | 0.912 | 0.749 | | 1.120 | 7.320 | +| ShuffleNetV2_swish | 0.700 | 0.892 | | | 0.290 | 2.260 | +| GhostNet_x0_5 | 0.668 | 0.869 | 0.662 | 0.866 | 0.082 | 2.600 | +| GhostNet_x1_0 | 0.740 | 0.916 | 0.739 | 0.914 | 0.294 | 5.200 | +| GhostNet_x1_3 | 0.757 | 0.925 | 0.757 | 0.927 | 0.440 | 7.300 | +| GhostNet_x1_3_ssld | 0.794 | 0.945 | 0.757 | 0.927 | 0.440 | 7.300 | + + + +## 3. 基于 SD855 的预测速度和存储大小 + +| Models | SD855 time(ms)
bs=1, thread=1 | SD855 time(ms)
bs=1, thread=2 | SD855 time(ms)
bs=1, thread=4 | Storage Size(M) | +|:--:|----|----|----|----| +| MobileNetV1_x0_25 | 2.88 | 1.82 | 1.26 | 1.900 | +| MobileNetV1_x0_5 | 8.74 | 5.26 | 3.09 | 5.200 | +| MobileNetV1_x0_75 | 17.84 | 10.61 | 6.21 | 10.000 | +| MobileNetV1 | 30.24 | 17.86 | 10.30 | 16.000 | +| MobileNetV1_ssld | 30.19 | 17.85 | 10.23 | 16.000 | +| MobileNetV2_x0_25 | 3.46 | 2.51 | 2.03 | 6.100 | +| MobileNetV2_x0_5 | 7.69 | 4.92 | 3.57 | 7.800 | +| MobileNetV2_x0_75 | 13.69 | 8.60 | 5.82 | 10.000 | +| MobileNetV2 | 20.74 | 12.71 | 8.10 | 14.000 | +| MobileNetV2_x1_5 | 40.79 | 24.49 | 15.50 | 26.000 | +| MobileNetV2_x2_0 | 67.50 | 40.03 | 25.55 | 43.000 | +| MobileNetV2_ssld | 20.71 | 12.70 | 8.06 | 14.000 | +| MobileNetV3_large_x1_25 | 24.52 | 14.76 | 9.89 | 29.000 | +| MobileNetV3_large_x1_0 | 16.55 | 10.09 | 6.84 | 21.000 | +| MobileNetV3_large_x0_75 | 11.53 | 7.06 | 4.94 | 16.000 | +| MobileNetV3_large_x0_5 | 6.50 | 4.22 | 3.15 | 11.000 | +| MobileNetV3_large_x0_35 | 4.43 | 3.11 | 2.41 | 8.600 | +| MobileNetV3_small_x1_25 | 7.88 | 4.91 | 3.45 | 14.000 | +| MobileNetV3_small_x1_0 | 5.63 | 3.65 | 2.60 | 12.000 | +| MobileNetV3_small_x0_75 | 4.50 | 2.96 | 2.19 | 9.600 | +| MobileNetV3_small_x0_5 | 2.89 | 2.04 | 1.62 | 7.800 | +| MobileNetV3_small_x0_35 | 2.23 | 1.66 | 1.43 | 6.900 | +| MobileNetV3_small_x0_35_ssld | | | | 6.900 | +| MobileNetV3_large_x1_0_ssld | 16.56 | 10.10 | 6.86 | 21.000 | +| MobileNetV3_large_x1_0_ssld_int8 | | | | 10.000 | +| MobileNetV3_small_x1_0_ssld | 5.64 | 3.67 | 2.61 | 12.000 | +| ShuffleNetV2 | 9.72 | 5.97 | 4.13 | 9.000 | +| ShuffleNetV2_x0_25 | 1.94 | 1.53 | 1.43 | 2.700 | +| ShuffleNetV2_x0_33 | 2.23 | 1.70 | 1.79 | 2.800 | +| ShuffleNetV2_x0_5 | 3.67 | 2.63 | 2.06 | 5.600 | +| ShuffleNetV2_x1_5 | 17.21 | 10.56 | 6.81 | 14.000 | +| ShuffleNetV2_x2_0 | 31.21 | 18.98 | 11.65 | 28.000 | +| ShuffleNetV2_swish | 31.21 | 9.06 | 5.74 | 9.100 | +| GhostNet_x0_5 | 5.28 | 3.95 | 3.29 | 10.000 | +| GhostNet_x1_0 | 12.89 | 8.66 | 6.72 | 20.000 | +| GhostNet_x1_3 | 19.16 | 12.25 | 9.40 | 29.000 | +| GhostNet_x1_3_ssld | 19.16 | 17.85 | 10.18 | 29.000 | + + + +## 4. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| -------------------------------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| MobileNetV1_x0_25 | 224 | 256 | 0.47 | 0.93 | 1.39 | +| MobileNetV1_x0_5 | 224 | 256 | 0.48 | 1.09 | 1.69 | +| MobileNetV1_x0_75 | 224 | 256 | 0.55 | 1.34 | 2.03 | +| MobileNetV1 | 224 | 256 | 0.64 | 1.57 | 2.48 | +| MobileNetV1_ssld | 224 | 256 | 0.66 | 1.59 | 2.58 | +| MobileNetV2_x0_25 | 224 | 256 | 0.83 | 1.17 | 1.78 | +| MobileNetV2_x0_5 | 224 | 256 | 0.84 | 1.45 | 2.04 | +| MobileNetV2_x0_75 | 224 | 256 | 0.96 | 1.62 | 2.53 | +| MobileNetV2 | 224 | 256 | 1.02 | 1.93 | 2.89 | +| MobileNetV2_x1_5 | 224 | 256 | 1.32 | 2.58 | 4.14 | +| MobileNetV2_x2_0 | 224 | 256 | 1.57 | 3.13 | 4.76 | +| MobileNetV2_ssld | 224 | 256 | 1.01 | 1.97 | 2.84 | +| MobileNetV3_large_x1_25 | 224 | 256 | 1.75 | 2.87 | 4.23 | +| MobileNetV3_large_x1_0 | 224 | 256 | 1.37 | 2.67 | 3.46 | +| MobileNetV3_large_x0_75 | 224 | 256 | 1.37 | 2.23 | 3.17 | +| MobileNetV3_large_x0_5 | 224 | 256 | 1.10 | 1.85 | 2.69 | +| MobileNetV3_large_x0_35 | 224 | 256 | 1.01 | 1.44 | 1.92 | +| MobileNetV3_small_x1_25 | 224 | 256 | 1.20 | 2.04 | 2.64 | +| MobileNetV3_small_x1_0 | 224 | 256 | 1.03 | 1.76 | 2.50 | +| MobileNetV3_small_x0_75 | 224 | 256 | 1.04 | 1.71 | 2.37 | +| MobileNetV3_small_x0_5 | 224 | 256 | 1.01 | 1.49 | 2.01 | +| MobileNetV3_small_x0_35 | 224 | 256 | 1.01 | 1.44 | 1.92 | +| MobileNetV3_small_x0_35_ssld | 224 | 256 | | | | +| MobileNetV3_large_x1_0_ssld | 224 | 256 | 1.35 | 2.47 | 3.72 | +| MobileNetV3_large_x1_0_ssld_int8 | 224 | 256 | | | | +| MobileNetV3_small_x1_0_ssld | 224 | 256 | 1.06 | 1.89 | 2.48 | +| ShuffleNetV2 | 224 | 256 | 1.05 | 1.76 | 2.37 | +| ShuffleNetV2_x0_25 | 224 | 256 | 0.92 | 1.27 | 1.73 | +| ShuffleNetV2_x0_33 | 224 | 256 | 0.91 | 1.29 | 1.81 | +| ShuffleNetV2_x0_5 | 224 | 256 | 0.89 | 1.43 | 1.94 | +| ShuffleNetV2_x1_5 | 224 | 256 | 0.93 | 1.99 | 2.85 | +| ShuffleNetV2_x2_0 | 224 | 256 | 1.45 | 2.70 | 3.35 | +| ShuffleNetV2_swish | 224 | 256 | 1.43 | 1.93 | 2.69 | +| GhostNet_x0_5 | 224 | 256 | 1.66 | 2.24 | 2.73 | +| GhostNet_x1_0 | 224 | 256 | 1.69 | 2.73 | 3.81 | +| GhostNet_x1_3 | 224 | 256 | 1.84 | 2.88 | 3.94 | +| GhostNet_x1_3_ssld | 224 | 256 | 1.85 | 3.17 | 4.29 | + + + +## 5. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-----------------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------| +| MobileNetV1_x0_25 | 224 | 256 | 0.47 | 0.93 | 1.39 | +| MobileNetV1_x0_5 | 224 | 256 | 0.48 | 1.09 | 1.69 | +| MobileNetV1_x0_75 | 224 | 256 | 0.55 | 1.34 | 2.03 | +| MobileNetV1 | 224 | 256 | 0.64 | 1.57 | 2.48 | +| MobileNetV1_ssld | 224 | 256 | 0.66 | 1.59 | 2.58 | +| MobileNetV2_x0_25 | 224 | 256 | 0.83 | 1.17 | 1.78 | +| MobileNetV2_x0_5 | 224 | 256 | 0.84 | 1.45 | 2.04 | +| MobileNetV2_x0_75 | 224 | 256 | 0.96 | 1.62 | 2.53 | +| MobileNetV2 | 224 | 256 | 1.02 | 1.93 | 2.89 | +| MobileNetV2_x1_5 | 224 | 256 | 1.32 | 2.58 | 4.14 | +| MobileNetV2_x2_0 | 224 | 256 | 1.57 | 3.13 | 4.76 | +| MobileNetV2_ssld | 224 | 256 | 1.01 | 1.97 | 2.84 | +| MobileNetV3_small_x0_35 | 224 | 256 | 1.01 | 1.44 | 1.92 | +| MobileNetV3_small_x0_5 | 224 | 256 | 1.01 | 1.49 | 2.01 | +| MobileNetV3_small_x0_75 | 224 | 256 | 1.04 | 1.71 | 2.37 | +| MobileNetV3_small_x1_0 | 224 | 256 | 1.03 | 1.76 | 2.50 | +| MobileNetV3_small_x1_25 | 224 | 256 | 1.20 | 2.04 | 2.64 | +| MobileNetV3_large_x0_35 | 224 | 256 | 1.10 | 1.74 | 2.34 | +| MobileNetV3_large_x0_5 | 224 | 256 | 1.10 | 1.85 | 2.69 | +| MobileNetV3_large_x0_75 | 224 | 256 | 1.37 | 2.23 | 3.17 | +| MobileNetV3_large_x1_0 | 224 | 256 | 1.37 | 2.67 | 3.46 | +| MobileNetV3_large_x1_25 | 224 | 256 | 1.75 | 2.87 | 4.23 | +| MobileNetV3_small_x1_0_ssld | 224 | 256 | 1.06 | 1.89 | 2.48 | +| MobileNetV3_large_x1_0_ssld | 224 | 256 | 1.35 | 2.47 | 3.72 | +| ShuffleNetV2_swish | 224 | 256 | 1.43 | 1.93 | 2.69 | +| ShuffleNetV2_x0_25 | 224 | 256 | 0.92 | 1.27 | 1.73 | +| ShuffleNetV2_x0_33 | 224 | 256 | 0.91 | 1.29 | 1.81 | +| ShuffleNetV2_x0_5 | 224 | 256 | 0.89 | 1.43 | 1.94 | +| ShuffleNetV2_x1_0 | 224 | 256 | 1.05 | 1.76 | 2.37 | +| ShuffleNetV2_x1_5 | 224 | 256 | 0.93 | 1.99 | 2.85 | +| ShuffleNetV2_x2_0 | 224 | 256 | 1.45 | 2.70 | 3.35 | +| GhostNet_x0_5 | 224 | 256 | 1.66 | 2.24 | 2.73 | +| GhostNet_x1_0 | 224 | 256 | 1.69 | 2.73 | 3.81 | +| GhostNet_x1_3 | 224 | 256 | 1.84 | 2.88 | 3.94 | +| GhostNet_x1_3_ssld | 224 | 256 | 1.85 | 3.17 | 4.29 | diff --git a/src/PaddleClas/docs/zh_CN/models/Others.md b/src/PaddleClas/docs/zh_CN/models/Others.md new file mode 100644 index 0000000..ff43c44 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/Others.md @@ -0,0 +1,64 @@ +# 其他模型 +----- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + + +## 1. 概述 +2012 年，Alex 等人提出的 AlexNet 网络在 ImageNet 大赛上以远超第二名的成绩夺冠，卷积神经网络乃至深度学习引起了广泛的关注。AlexNet 使用 relu 作为 CNN 的激活函数，解决了 sigmoid 在网络较深时的梯度弥散问题。训练时使用 Dropout 随机丢掉一部分神经元，避免了模型过拟合。网络中使用重叠的最大池化代替了此前 CNN 中普遍使用的平均池化，避免了平均池化的模糊效果，提升了特征的丰富性。从某种意义上说，AlexNet 引爆了神经网络的研究与应用热潮。 + +SqueezeNet 在 ImageNet-1k 上实现了与 AlexNet 相同的精度，但只用了 1/50 的参数量。该网络的核心是 Fire 模块，Fire 模块通过使用 1x1 的卷积实现通道降维，从而大大节省了参数量。作者通过大量堆叠 Fire 模块组成了 SqueezeNet。 + +VGG 由牛津大学计算机视觉组和 DeepMind 公司研究员一起研发的卷积神经网络。该网络探索了卷积神经网络的深度和其性能之间的关系，通过反复的堆叠 3x3 的小型卷积核和 2x2 的最大池化层，成功的构建了多层卷积神经网络并取得了不错的收敛精度。最终，VGG 获得了 ILSVRC 2014 比赛分类项目的亚军和定位项目的冠军。 + +DarkNet53 是 YOLO 作者在论文设计的用于目标检测的 backbone，该网络基本由 1x1 与 3x3 卷积构成，共 53 层，取名为 DarkNet53。 + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| AlexNet | 0.567 | 0.792 | 0.5720 | | 1.370 | 61.090 | +| SqueezeNet1_0 | 0.596 | 0.817 | 0.575 | | 1.550 | 1.240 | +| SqueezeNet1_1 | 0.601 | 0.819 | | | 0.690 | 1.230 | +| VGG11 | 0.693 | 0.891 | | | 15.090 | 132.850 | +| VGG13 | 0.700 | 0.894 | | | 22.480 | 133.030 | +| VGG16 | 0.720 | 0.907 | 0.715 | 0.901 | 30.810 | 138.340 | +| VGG19 | 0.726 | 0.909 | | | 39.130 | 143.650 | +| DarkNet53 | 0.780 | 0.941 | 0.772 | 0.938 | 18.580 | 41.600 | + + + +## 3. 基于 V100 GPU 的预测速度 + + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|---------------------------|-----------|-------------------|-------------------|-------------------|-------------------| +| AlexNet | 224 | 256 | 0.81 | 1.50 | 2.33 | +| SqueezeNet1_0 | 224 | 256 | 0.68 | 1.64 | 2.62 | +| SqueezeNet1_1 | 224 | 256 | 0.62 | 1.30 | 2.09 | +| VGG11 | 224 | 256 | 1.72 | 4.15 | 7.24 | +| VGG13 | 224 | 256 | 2.02 | 5.28 | 9.54 | +| VGG16 | 224 | 256 | 2.48 | 6.79 | 12.33 | +| VGG19 | 224 | 256 | 2.93 | 8.28 | 15.21 | +| DarkNet53 | 256 | 256 | 2.79 | 6.42 | 10.89 | + + + +## 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-----------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| AlexNet | 224 | 256 | 1.06447 | 1.70435 | 2.38402 | 1.44993 | 2.46696 | 3.72085 | +| SqueezeNet1_0 | 224 | 256 | 0.97162 | 2.06719 | 3.67499 | 0.96736 | 2.53221 | 4.54047 | +| SqueezeNet1_1 | 224 | 256 | 0.81378 | 1.62919 | 2.68044 | 0.76032 | 1.877 | 3.15298 | +| VGG11 | 224 | 256 | 2.24408 | 4.67794 | 7.6568 | 3.90412 | 9.51147 | 17.14168 | +| VGG13 | 224 | 256 | 2.58589 | 5.82708 | 10.03591 | 4.64684 | 12.61558 | 23.70015 | +| VGG16 | 224 | 256 | 3.13237 | 7.19257 | 12.50913 | 5.61769 | 16.40064 | 32.03939 | +| VGG19 | 224 | 256 | 3.69987 | 8.59168 | 15.07866 | 6.65221 | 20.4334 | 41.55902 | +| DarkNet53 | 256 | 256 | 3.18101 | 5.88419 | 10.14964 | 4.10829 | 12.1714 | 22.15266 | diff --git a/src/PaddleClas/docs/zh_CN/models/PP-LCNet.md b/src/PaddleClas/docs/zh_CN/models/PP-LCNet.md new file mode 100644 index 0000000..156f30e --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/PP-LCNet.md @@ -0,0 +1,204 @@ +# PP-LCNet 系列 +--- + +## 目录 + +- [1. 摘要](#1) +- [2. 介绍](#2) +- [3. 方法](#3) + - [3.1 更好的激活函数](#3.1) + - [3.2 合适的位置添加 SE 模块](#3.2) + - [3.3 合适的位置添加更大的卷积核](#3.3) + - [3.4 GAP 后使用更大的 1x1 卷积层](#3.4) +- [4. 实验部分](#4) + - [4.1 图像分类](#4.1) + - [4.2 目标检测](#4.2) + - [4.3 语义分割](#4.3) +- [5. 基于 V100 GPU 的预测速度](#5) +- [6. 基于 SD855 的预测速度](#6) +- [7. 总结](#7) +- [8. 引用](#8) + + + +## 1. 摘要 + +在计算机视觉领域中，骨干网络的好坏直接影响到整个视觉任务的结果。在之前的一些工作中，相关的研究者普遍将 FLOPs 或者 Params 作为优化目的，但是在工业界真实落地的场景中，推理速度才是考量模型好坏的重要指标，然而，推理速度和准确性很难兼得。考虑到工业界有很多基于 Intel CPU 的应用，所以我们本次的工作旨在使骨干网络更好的适应 Intel CPU，从而得到一个速度更快、准确率更高的轻量级骨干网络，与此同时，目标检测、语义分割等下游视觉任务的性能也同样得到提升。 + + +## 2. 介绍 + +近年来，有很多轻量级的骨干网络问世，尤其最近两年，各种 NAS 搜索出的网络层出不穷，这些网络要么主打 FLOPs 或者 Params 上的优势，要么主打 ARM 设备上的推理速度的优势，很少有网络专门针对 Intel CPU 做特定的优化，导致这些网络在 Intel CPU 端的推理速度并不是很完美。基于此，我们针对 Intel CPU 设备以及其加速库 MKLDNN 设计了特定的骨干网络 PP-LCNet，比起其他的轻量级的 SOTA 模型，该骨干网络可以在不增加推理时间的情况下，进一步提升模型的性能，最终大幅度超越现有的 SOTA 模型。与其他模型的对比图如下。 +![](../../images/PP-LCNet/PP-LCNet-Acc.png) + + +## 3. 方法 + +网络结构整体如下图所示。 +![](../../images/PP-LCNet/PP-LCNet.png) +我们经过大量的实验发现，在基于 Intel CPU 设备上，尤其当启用 MKLDNN 加速库后，很多看似不太耗时的操作反而会增加延时，比如 elementwise-add 操作、split-concat 结构等。所以最终我们选用了结构尽可能精简、速度尽可能快的 block 组成我们的 BaseNet（类似 MobileNetV1）。基于 BaseNet，我们通过实验，总结了四条几乎不增加延时但是可以提升模型精度的方法，融合这四条策略，我们组合成了 PP-LCNet。下面对这四条策略一一介绍： + + +### 3.1 更好的激活函数 + +自从卷积神经网络使用了 ReLU 激活函数后，网络性能得到了大幅度提升，近些年 ReLU 激活函数的变体也相继出现，如 Leaky-ReLU、P-ReLU、ELU 等，2017 年，谷歌大脑团队通过搜索的方式得到了 swish 激活函数，该激活函数在轻量级网络上表现优异，在 2019 年，MobileNetV3 的作者将该激活函数进一步优化为 H-Swish，该激活函数去除了指数运算，速度更快，网络精度几乎不受影响。我们也经过很多实验发现该激活函数在轻量级网络上有优异的表现。所以在 PP-LCNet 中，我们选用了该激活函数。 + + +### 3.2 合适的位置添加 SE 模块 + +SE 模块是 SENet 提出的一种通道注意力机制，可以有效提升模型的精度。但是在 Intel CPU 端，该模块同样会带来较大的延时，如何平衡精度和速度是我们要解决的一个问题。虽然在 MobileNetV3 等基于 NAS 搜索的网络中对 SE 模块的位置进行了搜索，但是并没有得出一般的结论，我们通过实验发现，SE 模块越靠近网络的尾部对模型精度的提升越大。下表也展示了我们的一些实验结果： + + +| SE Location | Top-1 Acc(\%) | Latency(ms) | +|-------------------|---------------|-------------| +| 1100000000000 | 61.73 | 2.06 | +| 0000001100000 | 62.17 | 2.03 | +| 0000000000011 | 63.14 | 2.05 | +| 1111111111111 | 64.27 | 3.80 | + + +最终，PP-LCNet 中的 SE 模块的位置选用了表格中第三行的方案。 + + +### 3.3 合适的位置添加更大的卷积核 + +在 MixNet 的论文中，作者分析了卷积核大小对模型性能的影响，结论是在一定范围内大的卷积核可以提升模型的性能，但是超过这个范围会有损模型的性能，所以作者组合了一种 split-concat 范式的 MixConv，这种组合虽然可以提升模型的性能，但是不利于推理。我们通过实验总结了一些更大的卷积核在不同位置的作用，类似 SE 模块的位置，更大的卷积核在网络的中后部作用更明显，下表展示了 5x5 卷积核的位置对精度的影响： + +| large-kernel Location | Top-1 Acc(\%) | Latency(ms) | +|-------------------|---------------|-------------| +| 1111111111111 | 63.22 | 2.08 | +| 1111111000000 | 62.70 | 2.07 | +| 0000001111111 | 63.14 | 2.05 | + + +实验表明，更大的卷积核放在网络的中后部即可达到放在所有位置的精度，与此同时，获得更快的推理速度。PP-LCNet 最终选用了表格中第三行的方案。 + + +### 3.4 GAP 后使用更大的 1x1 卷积层 + +在 GoogLeNet 之后，GAP（Global-Average-Pooling）后往往直接接分类层，但是在轻量级网络中，这样会导致 GAP 后提取的特征没有得到进一步的融合和加工。如果在此后使用一个更大的 1x1 卷积层（等同于 FC 层），GAP 后的特征便不会直接经过分类层，而是先进行了融合，并将融合的特征进行分类。这样可以在不影响模型推理速度的同时大大提升准确率。 +BaseNet 经过以上四个方面的改进，得到了 PP-LCNet。下表进一步说明了每个方案对结果的影响： + +| Activation | SE-block | Large-kernel | last-1x1-conv | Top-1 Acc(\%) | Latency(ms) | +|------------|----------|--------------|---------------|---------------|-------------| +| 0 | 1 | 1 | 1 | 61.93 | 1.94 | +| 1 | 0 | 1 | 1 | 62.51 | 1.87 | +| 1 | 1 | 0 | 1 | 62.44 | 2.01 | +| 1 | 1 | 1 | 0 | 59.91 | 1.85 | +| 1 | 1 | 1 | 1 | 63.14 | 2.05 | + + +## 4. 实验部分 + + +### 4.1 图像分类 + +图像分类我们选用了 ImageNet 数据集，相比目前主流的轻量级网络，PP-LCNet 在相同精度下可以获得更快的推理速度。当使用百度自研的 SSLD 蒸馏策略后，精度进一步提升，在 Intel cpu 端约 5ms 的推理速度下 ImageNet 的 Top-1 Acc 超过了 80%。 + +| Model | Params(M) | FLOPs(M) | Top-1 Acc(\%) | Top-5 Acc(\%) | Latency(ms) | +|-------|-----------|----------|---------------|---------------|-------------| +| PP-LCNet-0.25x | 1.5 | 18 | 51.86 | 75.65 | 1.74 | +| PP-LCNet-0.35x | 1.6 | 29 | 58.09 | 80.83 | 1.92 | +| PP-LCNet-0.5x | 1.9 | 47 | 63.14 | 84.66 | 2.05 | +| PP-LCNet-0.75x | 2.4 | 99 | 68.18 | 88.30 | 2.29 | +| PP-LCNet-1x | 3.0 | 161 | 71.32 | 90.03 | 2.46 | +| PP-LCNet-1.5x | 4.5 | 342 | 73.71 | 91.53 | 3.19 | +| PP-LCNet-2x | 6.5 | 590 | 75.18 | 92.27 | 4.27 | +| PP-LCNet-2.5x | 9.0 | 906 | 76.60 | 93.00 | 5.39 | +| PP-LCNet-0.5x\* | 1.9 | 47 | 66.10 | 86.46 | 2.05 | +| PP-LCNet-1.0x\* | 3.0 | 161 | 74.39 | 92.09 | 2.46 | +| PP-LCNet-2.5x\* | 9.0 | 906 | 80.82 | 95.33 | 5.39 | + +其中\*表示使用 SSLD 蒸馏后的模型。 + +与其他轻量级网络的性能对比： + +| Model | Params(M) | FLOPs(M) | Top-1 Acc(\%) | Top-5 Acc(\%) | Latency(ms) | +|-------|-----------|----------|---------------|---------------|-------------| +| MobileNetV2-0.25x | 1.5 | 34 | 53.21 | 76.52 | 2.47 | +| MobileNetV3-small-0.35x | 1.7 | 15 | 53.03 | 76.37 | 3.02 | +| ShuffleNetV2-0.33x | 0.6 | 24 | 53.73 | 77.05 | 4.30 | +| PP-LCNet-0.25x | 1.5 | 18 | 51.86 | 75.65 | 1.74 | +| MobileNetV2-0.5x | 2.0 | 99 | 65.03 | 85.72 | 2.85 | +| MobileNetV3-large-0.35x | 2.1 | 41 | 64.32 | 85.46 | 3.68 | +| ShuffleNetV2-0.5x | 1.4 | 43 | 60.32 | 82.26 | 4.65 | +| PP-LCNet-0.5x | 1.9 | 47 | 63.14 | 84.66 | 2.05 | +| MobileNetV1-1x | 4.3 | 578 | 70.99 | 89.68 | 3.38 | +| MobileNetV2-1x | 3.5 | 327 | 72.15 | 90.65 | 4.26 | +| MobileNetV3-small-1.25x | 3.6 | 100 | 70.67 | 89.51 | 3.95 | +| PP-LCNet-1x | 3.0 | 161 | 71.32 | 90.03 | 2.46 | + + +### 4.2 目标检测 + +目标检测的方法我们选用了百度自研的 PicoDet，该方法主打轻量级目标检测场景，下表展示了在 COCO 数据集上、backbone 选用 PP-LCNet 与 MobileNetV3 的结果的比较，无论在精度还是速度上，PP-LCNet 的优势都非常明显。 + +| Backbone | mAP(%) | Latency(ms) | +|-------|-----------|----------| +MobileNetV3-large-0.35x | 19.2 | 8.1 | +PP-LCNet-0.5x | 20.3 | 6.0 | +MobileNetV3-large-0.75x | 25.8 | 11.1 | +PP-LCNet-1x | 26.9 | 7.9 | + + +### 4.3 语义分割 + +语义分割的方法我们选用了 DeeplabV3+，下表展示了在 Cityscapes 数据集上、backbone 选用 PP-LCNet 与 MobileNetV3 的比较，在精度和速度方面，PP-LCNet 的优势同样明显。 + +| Backbone | mIoU(%) | Latency(ms) | +|-------|-----------|----------| +|MobileNetV3-large-0.5x | 55.42 | 135 | +|PP-LCNet-0.5x | 58.36 | 82 | +|MobileNetV3-large-0.75x | 64.53 | 151 | +|PP-LCNet-1x | 66.03 | 96 | + + + +## 5. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=1\4
(ms) | FP32
Batch Size=8
(ms) | +| ------------- | --------- | ----------------- | ---------------------------- | -------------------------------- | ------------------------------ | +| PPLCNet_x0_25 | 224 | 256 | 0.72 | 1.17 | 1.71 | +| PPLCNet_x0_35 | 224 | 256 | 0.69 | 1.21 | 1.82 | +| PPLCNet_x0_5 | 224 | 256 | 0.70 | 1.32 | 1.94 | +| PPLCNet_x0_75 | 224 | 256 | 0.71 | 1.49 | 2.19 | +| PPLCNet_x1_0 | 224 | 256 | 0.73 | 1.64 | 2.53 | +| PPLCNet_x1_5 | 224 | 256 | 0.82 | 2.06 | 3.12 | +| PPLCNet_x2_0 | 224 | 256 | 0.94 | 2.58 | 4.08 | + + + +## 6. 基于 SD855 的预测速度 + +| Models | SD855 time(ms)
bs=1, thread=1 | SD855 time(ms)
bs=1, thread=2 | SD855 time(ms)
bs=1, thread=4 | +| ------------- | -------------------------------- | --------------------------------- | --------------------------------- | +| PPLCNet_x0_25 | 2.30 | 1.62 | 1.32 | +| PPLCNet_x0_35 | 3.15 | 2.11 | 1.64 | +| PPLCNet_x0_5 | 4.27 | 2.73 | 1.92 | +| PPLCNet_x0_75 | 7.38 | 4.51 | 2.91 | +| PPLCNet_x1_0 | 10.78 | 6.49 | 3.98 | +| PPLCNet_x1_5 | 20.55 | 12.26 | 7.54 | +| PPLCNet_x2_0 | 33.79 | 20.17 | 12.10 | +| PPLCNet_x2_5 | 49.89 | 29.60 | 17.82 | + + + +## 7. 总结 + +PP-LCNet 没有像学术界那样死扣极致的 FLOPs 与 Params，而是着眼于分析如何添加对 Intel CPU 友好的模块来提升模型的性能，这样可以更好的平衡准确率和推理时间，其中的实验结论也很适合其他网络结构设计的研究者，同时也为 NAS 搜索研究者提供了更小的搜索空间和一般结论。最终的 PP-LCNet 在产业界也可以更好的落地和应用。 + + + +## 8. 引用 + +如果你的论文用到了 PP-LCNet 的方法，请添加如下 cite： +``` +@misc{cui2021pplcnet, + title={PP-LCNet: A Lightweight CPU Convolutional Neural Network}, + author={Cheng Cui and Tingquan Gao and Shengyu Wei and Yuning Du and Ruoyu Guo and Shuilong Dong and Bin Lu and Ying Zhou and Xueying Lv and Qiwen Liu and Xiaoguang Hu and Dianhai Yu and Yanjun Ma}, + year={2021}, + eprint={2109.15099}, + archivePrefix={arXiv}, + primaryClass={cs.CV} +} +``` diff --git a/src/PaddleClas/docs/zh_CN/models/PVTV2.md b/src/PaddleClas/docs/zh_CN/models/PVTV2.md new file mode 100644 index 0000000..0819a1d --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/PVTV2.md @@ -0,0 +1,27 @@ +# PVTV2 + +--- + +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) + + + +## 1. 概述 + +PVTV2 是 VisionTransformer 系列模型，该模型基于 PVT（Pyramid Vision Transformer）改进得到，PVT 模型使用 Transformer 结构构建了特征金字塔网络。PVTV2 的主要创新点有：1. 带 overlap 的 Patch embeding；2. 结合卷积神经网络；3. 注意力模块为线性复杂度。[论文地址](https://arxiv.org/pdf/2106.13797.pdf)。 + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| PVT_V2_B0 | 0.705 | 0.902 | 0.705 | - | 0.53 | 3.7 | +| PVT_V2_B1 | 0.787 | 0.945 | 0.787 | - | 2.0 | 14.0 | +| PVT_V2_B2 | 0.821 | 0.960 | 0.820 | - | 3.9 | 25.4 | +| PVT_V2_B3 | 0.831 | 0.965 | 0.831 | - | 6.7 | 45.2 | +| PVT_V2_B4 | 0.836 | 0.967 | 0.836 | - | 9.8 | 62.6 | +| PVT_V2_B5 | 0.837 | 0.966 | 0.838 | - | 11.4 | 82.0 | +| PVT_V2_B2_Linear | 0.821 | 0.961 | 0.821 | - | 3.8 | 22.6 | diff --git a/src/PaddleClas/docs/zh_CN/models/ReXNet.md b/src/PaddleClas/docs/zh_CN/models/ReXNet.md new file mode 100644 index 0000000..37e93fd --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/ReXNet.md @@ -0,0 +1,40 @@ +# ReXNet 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) + + + +## 1. 概述 + +ReXNet 是 NAVER 集团 ClovaAI 研发中心基于一种网络架构设计新范式而构建的网络。针对现有网络中存在的 `Representational Bottleneck` 问题，作者提出了一组新的设计原则。作者认为传统的网络架构设计范式会产生表达瓶颈，进而影响模型的性能。为研究此问题，作者研究了上万个随机网络生成特征的 `matric rank`，同时进一步研究了网络层中通道配置方案。基于此，作者提出了一组简单而有效的设计原则，以消除表达瓶颈问题。[论文地址](https://arxiv.org/pdf/2007.00992.pdf) + + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1| FLOPS
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|----| +| ReXNet_1_0 | 77.46 | 93.70 | 77.9 | 0.415 | 4.838 | +| ReXNet_1_3 | 79.13 | 94.64 | 79.5 | 0.683 | 7.611 | +| ReXNet_1_5 | 80.06 | 95.12 | 80.3 | 0.900 | 9.791 | +| ReXNet_2_0 | 81.22 | 95.36 | 81.6 | 1.561 | 16.449 | +| ReXNet_3_0 | 82.09 | 96.12 | 82.8 | 3.445 | 34.833 | + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| ---------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| ReXNet_1_0 | 224 | 256 | 3.08 | 4.15 | 5.49 | +| ReXNet_1_3 | 224 | 256 | 3.54 | 4.87 | 6.54 | +| ReXNet_1_5 | 224 | 256 | 3.68 | 5.31 | 7.38 | +| ReXNet_2_0 | 224 | 256 | 4.30 | 6.54 | 9.19 | +| ReXNet_3_0 | 224 | 256 | 5.74 | 9.49 | 13.62 | + +关于 Inference speed 等信息，敬请期待。 diff --git a/src/PaddleClas/docs/zh_CN/models/RedNet.md b/src/PaddleClas/docs/zh_CN/models/RedNet.md new file mode 100644 index 0000000..2f75060 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/RedNet.md @@ -0,0 +1,35 @@ +# RedNet 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) + + +## 1. 概述 + +在 ResNet 的 Backbone 和 Backbone 的所有 Bottleneck 位置上使用 Involution 替换掉了卷积，但保留了所有的卷积用于通道映射和融合。这些精心重新设计的实体联合起来，形成了一种新的高效 Backbone 网络，称为 RedNet。[论文地址](https://arxiv.org/abs/2103.06255)。 + + +## 2. 精度、FLOPS 和参数量 + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:---------------------:|:----------:|:---------:|:---------:|:---------:| +| RedNet26 | 9.2 | 1.7 | 75.95 | 93.19 | +| RedNet38 | 12.4 | 2.2 | 77.47 | 93.56 | +| RedNet50 | 15.5 | 2.7 | 78.33 | 94.17 | +| RedNet101 | 25.7 | 4.7 | 78.94 | 94.36 | +| RedNet152 | 34.0 | 6.8 | 79.17 | 94.40 | + + + +## 3. 基于 V100 GPU 的预测速度 + +| 模型 | Crop Size | Resize Short Size | time(ms)
bs=1 | time(ms)
bs=4 | time(ms)
bs=8 | +| --------- | --------- | ----------------- | ---------------- | ---------------- | ----------------- | +| RedNet26 | 224 | 256 | 4.45 | 15.16 | 29.03 | +| RedNet38 | 224 | 256 | 6.24 | 21.39 | 41.26 | +| RedNet50 | 224 | 256 | 8.04 | 27.71 | 53.73 | +| RedNet101 | 224 | 256 | 13.07 | 44.12 | 83.28 | +| RedNet152 | 224 | 256 | 18.66 | 63.27 | 119.48 | \ No newline at end of file diff --git a/src/PaddleClas/docs/zh_CN/models/RepVGG.md b/src/PaddleClas/docs/zh_CN/models/RepVGG.md new file mode 100644 index 0000000..1ef9765 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/RepVGG.md @@ -0,0 +1,29 @@ +# RepVGG 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) + + +## 1. 概述 + +RepVGG(Making VGG-style ConvNets Great Again)系列模型是由清华大学(丁贵广团队)、旷视科技(孙剑等人)、港科大和阿伯里斯特威斯大学在 2021 年提出的一个简单但强大的卷积神经网络架构，该架构具有类似于 VGG 的推理时间主体，该主体仅由 3x3 卷积和 ReLU 的堆栈组成，而训练时间模型具有多分支拓扑。训练时间和推理时间架构的这种解耦是通过结构重新参数化(re-parameterization)技术实现的，因此该模型称为 RepVGG。[论文地址](https://arxiv.org/abs/2101.03697)。 + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1| FLOPS
(G) | +|:--:|:--:|:--:|:--:|:--:| +| RepVGG_A0 | 0.7131 | 0.9016 | 0.7241 | | +| RepVGG_A1 | 0.7380 | 0.9146 | 0.7446 | | +| RepVGG_A2 | 0.7571 | 0.9264 | 0.7648 | | +| RepVGG_B0 | 0.7450 | 0.9213 | 0.7514 | | +| RepVGG_B1 | 0.7773 | 0.9385 | 0.7837 | | +| RepVGG_B2 | 0.7813 | 0.9410 | 0.7878 | | +| RepVGG_B1g2 | 0.7732 | 0.9359 | 0.7778 | | +| RepVGG_B1g4 | 0.7675 | 0.9335 | 0.7758 | | +| RepVGG_B2g4 | 0.7881 | 0.9448 | 0.7938 | | +| RepVGG_B3g4 | 0.7965 | 0.9485 | 0.8021 | | + +关于 Params、FLOPs、Inference speed 等信息，敬请期待。 diff --git a/src/PaddleClas/docs/zh_CN/models/ResNeSt_RegNet.md b/src/PaddleClas/docs/zh_CN/models/ResNeSt_RegNet.md new file mode 100644 index 0000000..967351b --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/ResNeSt_RegNet.md @@ -0,0 +1,46 @@ +# ResNeSt 与 RegNet 系列 +---- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + + +## 1. 概述 + +ResNeSt 系列模型是在 2020 年提出的，在原有的 resnet 网络结构上做了改进，通过引入 K 个 Group 和在不同 Group 中加入类似于 SEBlock 的 attention 模块，使得精度相比于基础模型 ResNet 有了大幅度的提高，且参数量和 flops 与基础的 ResNet 基本保持一致。 + +RegNet 是由 facebook 于 2020 年提出，旨在深化设计空间理念的概念，在 AnyNetX 的基础上逐步改进，通过加入共享瓶颈 ratio、共享组宽度、调整网络深度与宽度等策略，最终实现简化设计空间结构、提高设计空间的可解释性、改善设计空间的质量，并保持设计空间的模型多样性的目的。最终设计出的模型在类似的条件下，性能还要优于 EfficientNet，并且在 GPU 上的速度提高了 5 倍。 + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ResNeSt50_fast_1s1x64d | 0.8035 | 0.9528| 0.8035 | -| 8.68 | 26.3 | +| ResNeSt50 | 0.8083 | 0.9542| 0.8113 | -| 10.78 | 27.5 | +| RegNetX_4GF | 0.7850 | 0.9416| 0.7860 | -| 8.0 | 22.1 | + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| ---------------------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| ResNeSt50_fast_1s1x64d | 224 | 256 | 2.73 | 5.33 | 8.24 | +| ResNeSt50 | 224 | 256 | 7.36 | 10.23 | 13.84 | +| RegNetX_4GF | 224 | 256 | 6.46 | 8.48 | 11.45 | + + + +## 4. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|--------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| ResNeSt50_fast_1s1x64d | 224 | 256 | 3.46466 | 5.56647 | 9.11848 | 3.45405 | 8.72680 | 15.48710 | +| ResNeSt50 | 224 | 256 | 7.05851 | 8.97676 | 13.34704 | 6.16248 | 12.0633 | 21.49936 | +| RegNetX_4GF | 224 | 256 | 6.69042 | 8.01664 | 11.60608 | 6.46478 | 11.19862 | 16.89089 | diff --git a/src/PaddleClas/docs/zh_CN/models/ResNet_and_vd.md b/src/PaddleClas/docs/zh_CN/models/ResNet_and_vd.md new file mode 100644 index 0000000..3e8e600 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/ResNet_and_vd.md @@ -0,0 +1,108 @@ +# ResNet 及其 Vd 系列 +----- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + + +## 1. 概述 + +ResNet 系列模型是在 2015 年提出的，一举在 ILSVRC2015 比赛中取得冠军，top5 错误率为 3.57%。该网络创新性的提出了残差结构，通过堆叠多个残差结构从而构建了 ResNet 网络。实验表明使用残差块可以有效地提升收敛速度和精度。 + +斯坦福大学的 Joyce Xu 将 ResNet 称为「真正重新定义了我们看待神经网络的方式」的三大架构之一。由于 ResNet 卓越的性能，越来越多的来自学术界和工业界学者和工程师对其结构进行了改进，比较出名的有 Wide-ResNet, ResNet-vc,ResNet-vd, Res2Net 等，其中 ResNet-vc 与 ResNet-vd 的参数量和计算量与 ResNet 几乎一致，所以在此我们将其与 ResNet 统一归为 ResNet 系列。 + +本次发布 ResNet 系列的模型包括 ResNet50，ResNet50_vd，ResNet50_vd_ssld，ResNet200_vd 等 14 个预训练模型。在训练层面上，ResNet 的模型采用了训练 ImageNet 的标准训练流程，而其余改进版模型采用了更多的训练策略，如 learning rate 的下降方式采用了 cosine decay，引入了 label smoothing 的标签正则方式，在数据预处理加入了 mixup 的操作，迭代总轮数从 120 个 epoch 增加到 200 个 epoch。 + +其中，ResNet50_vd_v2 与 ResNet50_vd_ssld 采用了知识蒸馏，保证模型结构不变的情况下，进一步提升了模型的精度，具体地，ResNet50_vd_v2 的 teacher 模型是 ResNet152_vd（top1 准确率 80.59%），数据选用的是 ImageNet-1k 的训练集，ResNet50_vd_ssld 的 teacher 模型是 ResNeXt101_32x16d_wsl（top1 准确率 84.2%），数据选用结合了 ImageNet-1k 的训练集和 ImageNet-22k 挖掘的 400 万数据。知识蒸馏的具体方法正在持续更新中。 + +该系列模型的 FLOPS、参数量以及 T4 GPU 上的预测耗时如下图所示。 + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.ResNet.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.ResNet.png) + + +通过上述曲线可以看出，层数越多，准确率越高，但是相应的参数量、计算量和延时都会增加。ResNet50_vd_ssld 通过用更强的 teacher 和更多的数据，将其在 ImageNet-1k 上的验证集 top-1 精度进一步提高，达到了 82.39%，刷新了 ResNet50 系列模型的精度。 + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ResNet18 | 0.710 | 0.899 | 0.696 | 0.891 | 3.660 | 11.690 | +| ResNet18_vd | 0.723 | 0.908 | | | 4.140 | 11.710 | +| ResNet34 | 0.746 | 0.921 | 0.732 | 0.913 | 7.360 | 21.800 | +| ResNet34_vd | 0.760 | 0.930 | | | 7.390 | 21.820 | +| ResNet34_vd_ssld | 0.797 | 0.949 | | | 7.390 | 21.820 | +| ResNet50 | 0.765 | 0.930 | 0.760 | 0.930 | 8.190 | 25.560 | +| ResNet50_vc | 0.784 | 0.940 | | | 8.670 | 25.580 | +| ResNet50_vd | 0.791 | 0.944 | 0.792 | 0.946 | 8.670 | 25.580 | +| ResNet50_vd_v2 | 0.798 | 0.949 | | | 8.670 | 25.580 | +| ResNet101 | 0.776 | 0.936 | 0.776 | 0.938 | 15.520 | 44.550 | +| ResNet101_vd | 0.802 | 0.950 | | | 16.100 | 44.570 | +| ResNet152 | 0.783 | 0.940 | 0.778 | 0.938 | 23.050 | 60.190 | +| ResNet152_vd | 0.806 | 0.953 | | | 23.530 | 60.210 | +| ResNet200_vd | 0.809 | 0.953 | | | 30.530 | 74.740 | +| ResNet50_vd_ssld | 0.824 | 0.961 | | | 8.670 | 25.580 | +| ResNet50_vd_ssld_v2 | 0.830 | 0.964 | | | 8.670 | 25.580 | +| Fix_ResNet50_vd_ssld_v2 | 0.840 | 0.970 | | | 17.696 | 25.580 | +| ResNet101_vd_ssld | 0.837 | 0.967 | | | 16.100 | 44.570 | + +* 注：`ResNet50_vd_ssld_v2` 是在 `ResNet50_vd_ssld` 训练策略的基础上加上 AutoAugment 训练得到，`Fix_ResNet50_vd_ssld_v2` 是固定 `ResNet50_vd_ssld_v2` 除 FC 层外所有的网络参数，在 320x320 的图像输入分辨率下，基于 ImageNet1k 数据集微调得到。 + + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=1\4
(ms) | FP32
Batch Size=8
(ms) | +|------------------|-----------|-------------------|--------------------------|--------------------------|--------------------------| +| ResNet18 | 224 | 256 | 1.22 | 2.19 | 3.63 | +| ResNet18_vd | 224 | 256 | 1.26 | 2.28 | 3.89 | +| ResNet34 | 224 | 256 | 1.97 | 3.25 | 5.70 | +| ResNet34_vd | 224 | 256 | 2.00 | 3.28 | 5.84 | +| ResNet34_vd_ssld | 224 | 256 | 2.00 | 3.26 | 5.85 | +| ResNet50 | 224 | 256 | 2.54 | 4.79 | 7.40 | +| ResNet50_vc | 224 | 256 | 2.57 | 4.83 | 7.52 | +| ResNet50_vd | 224 | 256 | 2.60 | 4.86 | 7.63 | +| ResNet50_vd_v2 | 224 | 256 | 2.59 | 4.86 | 7.59 | +| ResNet101 | 224 | 256 | 4.37 | 8.18 | 12.38 | +| ResNet101_vd | 224 | 256 | 4.43 | 8.25 | 12.60 | +| ResNet152 | 224 | 256 | 6.05 | 11.41 | 17.33 | +| ResNet152_vd | 224 | 256 | 6.11 | 11.51 | 17.59 | +| ResNet200_vd | 224 | 256 | 7.70 | 14.57 | 22.16 | +| ResNet50_vd_ssld | 224 | 256 | 2.59 | 4.87 | 7.62 | +| ResNet101_vd_ssld | 224 | 256 | 4.43 | 8.25 | 12.58 | + + + +## 4. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| ResNet18 | 224 | 256 | 1.3568 | 2.5225 | 3.61904 | 1.45606 | 3.56305 | 6.28798 | +| ResNet18_vd | 224 | 256 | 1.39593 | 2.69063 | 3.88267 | 1.54557 | 3.85363 | 6.88121 | +| ResNet34 | 224 | 256 | 2.23092 | 4.10205 | 5.54904 | 2.34957 | 5.89821 | 10.73451 | +| ResNet34_vd | 224 | 256 | 2.23992 | 4.22246 | 5.79534 | 2.43427 | 6.22257 | 11.44906 | +| ResNet34_vd_ssld | 224 | 256 | 2.23992 | 4.22246 | 5.79534 | 2.43427 | 6.22257 | 11.44906 | +| ResNet50 | 224 | 256 | 2.63824 | 4.63802 | 7.02444 | 3.47712 | 7.84421 | 13.90633 | +| ResNet50_vc | 224 | 256 | 2.67064 | 4.72372 | 7.17204 | 3.52346 | 8.10725 | 14.45577 | +| ResNet50_vd | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| ResNet50_vd_v2 | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| ResNet101 | 224 | 256 | 5.04037 | 7.73673 | 10.8936 | 6.07125 | 13.40573 | 24.3597 | +| ResNet101_vd | 224 | 256 | 5.05972 | 7.83685 | 11.34235 | 6.11704 | 13.76222 | 25.11071 | +| ResNet152 | 224 | 256 | 7.28665 | 10.62001 | 14.90317 | 8.50198 | 19.17073 | 35.78384 | +| ResNet152_vd | 224 | 256 | 7.29127 | 10.86137 | 15.32444 | 8.54376 | 19.52157 | 36.64445 | +| ResNet200_vd | 224 | 256 | 9.36026 | 13.5474 | 19.0725 | 10.80619 | 25.01731 | 48.81399 | +| ResNet50_vd_ssld | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| ResNet50_vd_ssld_v2 | 224 | 256 | 2.65164 | 4.84109 | 7.46225 | 3.53131 | 8.09057 | 14.45965 | +| Fix_ResNet50_vd_ssld_v2 | 320 | 320 | 3.42818 | 7.51534 | 13.19370 | 5.07696 | 14.64218 | 27.01453 | +| ResNet101_vd_ssld | 224 | 256 | 5.05972 | 7.83685 | 11.34235 | 6.11704 | 13.76222 | 25.11071 | diff --git a/src/PaddleClas/docs/zh_CN/models/SEResNext_and_Res2Net.md b/src/PaddleClas/docs/zh_CN/models/SEResNext_and_Res2Net.md new file mode 100644 index 0000000..30fac65 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/SEResNext_and_Res2Net.md @@ -0,0 +1,134 @@ +# SEResNeXt 与 Res2Net 系列 + +----- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于 V100 GPU 的预测速度](#3) +* [4. 基于 T4 GPU 的预测速度](#4) + + + +## 1. 概述 +ResNeXt 是 ResNet 的典型变种网络之一，ResNeXt 发表于 2017 年的 CVPR 会议。在此之前，提升模型精度的方法主要集中在将网络变深或者变宽，这样增加了参数量和计算量，推理速度也会相应变慢。ResNeXt 结构提出了通道分组（cardinality）的概念，作者通过实验发现增加通道的组数比增加深度和宽度更有效。其可以在不增加参数复杂度的前提下提高准确率，同时还减少了参数的数量，所以是比较成功的 ResNet 的变种。 + +SENet 是 2017 年 ImageNet 分类比赛的冠军方案，其提出了一个全新的 SE 结构，该结构可以迁移到任何其他网络中，其通过控制 scale 的大小，把每个通道间重要的特征增强，不重要的特征减弱，从而让提取的特征指向性更强。 + +Res2Net 是 2019 年提出的一种全新的对 ResNet 的改进方案，该方案可以和现有其他优秀模块轻松整合，在不增加计算负载量的情况下，在 ImageNet、CIFAR-100 等数据集上的测试性能超过了 ResNet。Res2Net 结构简单，性能优越，进一步探索了 CNN 在更细粒度级别的多尺度表示能力。Res2Net 揭示了一个新的提升模型精度的维度，即 scale，其是除了深度、宽度和基数的现有维度之外另外一个必不可少的更有效的因素。该网络在其他视觉任务如目标检测、图像分割等也有相当不错的表现。 + +该系列模型的 FLOPS、参数量以及 T4 GPU 上的预测耗时如下图所示。 + + +![](../../images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.flops.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.params.png) + +![](../../images/models/T4_benchmark/t4.fp32.bs4.SeResNeXt.png) + +![](../../images/models/T4_benchmark/t4.fp16.bs4.SeResNeXt.png) + + +目前 PaddleClas 开源的这三类的预训练模型一共有 24 个，其指标如图所示，从图中可以看出，在同样 Flops 和 Params 下，改进版的模型往往有更高的精度，但是推理速度往往不如 ResNet 系列。另一方面，Res2Net 表现也较为优秀，相比 ResNeXt 中的 group 操作、SEResNet 中的 SE 结构操作，Res2Net 在相同 Flops、Params 和推理速度下往往精度更佳。 + + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Parameters
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| Res2Net50_26w_4s | 0.793 | 0.946 | 0.780 | 0.936 | 8.520 | 25.700 | +| Res2Net50_vd_26w_4s | 0.798 | 0.949 | | | 8.370 | 25.060 | +| Res2Net50_vd_26w_4s_ssld | 0.831 | 0.966 | | | 8.370 | 25.060 | +| Res2Net50_14w_8s | 0.795 | 0.947 | 0.781 | 0.939 | 9.010 | 25.720 | +| Res2Net101_vd_26w_4s | 0.806 | 0.952 | | | 16.670 | 45.220 | +| Res2Net101_vd_26w_4s_ssld | 0.839 | 0.971 | | | 16.670 | 45.220 | +| Res2Net200_vd_26w_4s | 0.812 | 0.957 | | | 31.490 | 76.210 | +| Res2Net200_vd_26w_4s_ssld | **0.851** | 0.974 | | | 31.490 | 76.210 | +| ResNeXt50_32x4d | 0.778 | 0.938 | 0.778 | | 8.020 | 23.640 | +| ResNeXt50_vd_32x4d | 0.796 | 0.946 | | | 8.500 | 23.660 | +| ResNeXt50_64x4d | 0.784 | 0.941 | | | 15.060 | 42.360 | +| ResNeXt50_vd_64x4d | 0.801 | 0.949 | | | 15.540 | 42.380 | +| ResNeXt101_32x4d | 0.787 | 0.942 | 0.788 | | 15.010 | 41.540 | +| ResNeXt101_vd_32x4d | 0.803 | 0.951 | | | 15.490 | 41.560 | +| ResNeXt101_64x4d | 0.784 | 0.945 | 0.796 | | 29.050 | 78.120 | +| ResNeXt101_vd_64x4d | 0.808 | 0.952 | | | 29.530 | 78.140 | +| ResNeXt152_32x4d | 0.790 | 0.943 | | | 22.010 | 56.280 | +| ResNeXt152_vd_32x4d | 0.807 | 0.952 | | | 22.490 | 56.300 | +| ResNeXt152_64x4d | 0.795 | 0.947 | | | 43.030 | 107.570 | +| ResNeXt152_vd_64x4d | 0.811 | 0.953 | | | 43.520 | 107.590 | +| SE_ResNet18_vd | 0.733 | 0.914 | | | 4.140 | 11.800 | +| SE_ResNet34_vd | 0.765 | 0.932 | | | 7.840 | 21.980 | +| SE_ResNet50_vd | 0.795 | 0.948 | | | 8.670 | 28.090 | +| SE_ResNeXt50_32x4d | 0.784 | 0.940 | 0.789 | 0.945 | 8.020 | 26.160 | +| SE_ResNeXt50_vd_32x4d | 0.802 | 0.949 | | | 10.760 | 26.280 | +| SE_ResNeXt101_32x4d | 0.7939 | 0.9443 | 0.793 | 0.950 | 15.020 | 46.280 | +| SENet154_vd | 0.814 | 0.955 | | | 45.830 | 114.290 | + + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-----------------------|-----------|-------------------|-----------------------|-----------------------|-----------------------| +| Res2Net50_26w_4s | 224 | 256 | 3.52 | 6.23 | 9.30 | +| Res2Net50_vd_26w_4s | 224 | 256 | 3.59 | 6.35 | 9.50 | +| Res2Net50_14w_8s | 224 | 256 | 4.39 | 7.21 | 10.38 | +| Res2Net101_vd_26w_4s | 224 | 256 | 6.34 | 11.02 | 16.13 | +| Res2Net200_vd_26w_4s | 224 | 256 | 11.45 | 19.77 | 28.81 | +| ResNeXt50_32x4d | 224 | 256 | 5.07 | 8.49 | 12.02 | +| ResNeXt50_vd_32x4d | 224 | 256 | 5.29 | 8.68 | 12.33 | +| ResNeXt50_64x4d | 224 | 256 | 9.39 | 13.97 | 20.56 | +| ResNeXt50_vd_64x4d | 224 | 256 | 9.75 | 14.14 | 20.84 | +| ResNeXt101_32x4d | 224 | 256 | 11.34 | 16.78 | 22.80 | +| ResNeXt101_vd_32x4d | 224 | 256 | 11.36 | 17.01 | 23.07 | +| ResNeXt101_64x4d | 224 | 256 | 21.57 | 28.08 | 39.49 | +| ResNeXt101_vd_64x4d | 224 | 256 | 21.57 | 28.22 | 39.70 | +| ResNeXt152_32x4d | 224 | 256 | 17.14 | 25.11 | 33.79 | +| ResNeXt152_vd_32x4d | 224 | 256 | 16.99 | 25.29 | 33.85 | +| ResNeXt152_64x4d | 224 | 256 | 33.07 | 42.05 | 59.13 | +| ResNeXt152_vd_64x4d | 224 | 256 | 33.30 | 42.41 | 59.42 | +| SE_ResNet18_vd | 224 | 256 | 1.48 | 2.70 | 4.32 | +| SE_ResNet34_vd | 224 | 256 | 2.42 | 3.69 | 6.29 | +| SE_ResNet50_vd | 224 | 256 | 3.11 | 5.99 | 9.34 | +| SE_ResNeXt50_32x4d | 224 | 256 | 6.39 | 11.01 | 14.94 | +| SE_ResNeXt50_vd_32x4d | 224 | 256 | 7.04 | 11.57 | 16.01 | +| SE_ResNeXt101_32x4d | 224 | 256 | 13.31 | 21.85 | 28.77 | +| SENet154_vd | 224 | 256 | 34.83 | 51.22 | 69.74 | +| Res2Net50_vd_26w_4s_ssld | 224 | 256 | 3.58 | 6.35 | 9.52 | +| Res2Net101_vd_26w_4s_ssld | 224 | 256 | 6.33 | 11.02 | 16.11 | +| Res2Net200_vd_26w_4s_ssld | 224 | 256 | 11.47 | 19.75 | 28.83 | + + + + +## 4. 基于 T4 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP16
Batch Size=1
(ms) | FP16
Batch Size=4
(ms) | FP16
Batch Size=8
(ms) | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +|-----------------------|-----------|-------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------|------------------------------| +| Res2Net50_26w_4s | 224 | 256 | 3.56067 | 6.61827 | 11.41566 | 4.47188 | 9.65722 | 17.54535 | +| Res2Net50_vd_26w_4s | 224 | 256 | 3.69221 | 6.94419 | 11.92441 | 4.52712 | 9.93247 | 18.16928 | +| Res2Net50_14w_8s | 224 | 256 | 4.45745 | 7.69847 | 12.30935 | 5.4026 | 10.60273 | 18.01234 | +| Res2Net101_vd_26w_4s | 224 | 256 | 6.53122 | 10.81895 | 18.94395 | 8.08729 | 17.31208 | 31.95762 | +| Res2Net200_vd_26w_4s | 224 | 256 | 11.66671 | 18.93953 | 33.19188 | 14.67806 | 32.35032 | 63.65899 | +| ResNeXt50_32x4d | 224 | 256 | 7.61087 | 8.88918 | 12.99674 | 7.56327 | 10.6134 | 18.46915 | +| ResNeXt50_vd_32x4d | 224 | 256 | 7.69065 | 8.94014 | 13.4088 | 7.62044 | 11.03385 | 19.15339 | +| ResNeXt50_64x4d | 224 | 256 | 13.78688 | 15.84655 | 21.79537 | 13.80962 | 18.4712 | 33.49843 | +| ResNeXt50_vd_64x4d | 224 | 256 | 13.79538 | 15.22201 | 22.27045 | 13.94449 | 18.88759 | 34.28889 | +| ResNeXt101_32x4d | 224 | 256 | 16.59777 | 17.93153 | 21.36541 | 16.21503 | 19.96568 | 33.76831 | +| ResNeXt101_vd_32x4d | 224 | 256 | 16.36909 | 17.45681 | 22.10216 | 16.28103 | 20.25611 | 34.37152 | +| ResNeXt101_64x4d | 224 | 256 | 30.12355 | 32.46823 | 38.41901 | 30.4788 | 36.29801 | 68.85559 | +| ResNeXt101_vd_64x4d | 224 | 256 | 30.34022 | 32.27869 | 38.72523 | 30.40456 | 36.77324 | 69.66021 | +| ResNeXt152_32x4d | 224 | 256 | 25.26417 | 26.57001 | 30.67834 | 24.86299 | 29.36764 | 52.09426 | +| ResNeXt152_vd_32x4d | 224 | 256 | 25.11196 | 26.70515 | 31.72636 | 25.03258 | 30.08987 | 52.64429 | +| ResNeXt152_64x4d | 224 | 256 | 46.58293 | 48.34563 | 56.97961 | 46.7564 | 56.34108 | 106.11736 | +| ResNeXt152_vd_64x4d | 224 | 256 | 47.68447 | 48.91406 | 57.29329 | 47.18638 | 57.16257 | 107.26288 | +| SE_ResNet18_vd | 224 | 256 | 1.61823 | 3.1391 | 4.60282 | 1.7691 | 4.19877 | 7.5331 | +| SE_ResNet34_vd | 224 | 256 | 2.67518 | 5.04694 | 7.18946 | 2.88559 | 7.03291 | 12.73502 | +| SE_ResNet50_vd | 224 | 256 | 3.65394 | 7.568 | 12.52793 | 4.28393 | 10.38846 | 18.33154 | +| SE_ResNeXt50_32x4d | 224 | 256 | 9.06957 | 11.37898 | 18.86282 | 8.74121 | 13.563 | 23.01954 | +| SE_ResNeXt50_vd_32x4d | 224 | 256 | 9.25016 | 11.85045 | 25.57004 | 9.17134 | 14.76192 | 19.914 | +| SE_ResNeXt101_32x4d | 224 | 256 | 19.34455 | 20.6104 | 32.20432 | 18.82604 | 25.31814 | 41.97758 | +| SENet154_vd | 224 | 256 | 49.85733 | 54.37267 | 74.70447 | 53.79794 | 66.31684 | 121.59885 | diff --git a/src/PaddleClas/docs/zh_CN/models/SwinTransformer.md b/src/PaddleClas/docs/zh_CN/models/SwinTransformer.md new file mode 100644 index 0000000..40a8732 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/SwinTransformer.md @@ -0,0 +1,48 @@ +# SwinTransformer +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于V100 GPU 的预测速度](#3) + + + +## 1. 概述 +Swin Transformer 是一种新的视觉 Transformer 网络，可以用作计算机视觉领域的通用骨干网路。SwinTransformer 由移动窗口（shifted windows）表示的层次 Transformer 结构组成。移动窗口将自注意计算限制在非重叠的局部窗口上，同时允许跨窗口连接，从而提高了网络性能。[论文地址](https://arxiv.org/abs/2103.14030)。 + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| SwinTransformer_tiny_patch4_window7_224 | 0.8069 | 0.9534 | 0.812 | 0.955 | 4.5 | 28 | +| SwinTransformer_small_patch4_window7_224 | 0.8275 | 0.9613 | 0.832 | 0.962 | 8.7 | 50 | +| SwinTransformer_base_patch4_window7_224 | 0.8300 | 0.9626 | 0.835 | 0.965 | 15.4 | 88 | +| SwinTransformer_base_patch4_window12_384 | 0.8439 | 0.9693 | 0.845 | 0.970 | 47.1 | 88 | +| SwinTransformer_base_patch4_window7_224[1] | 0.8487 | 0.9746 | 0.852 | 0.975 | 15.4 | 88 | +| SwinTransformer_base_patch4_window12_384[1] | 0.8642 | 0.9807 | 0.864 | 0.980 | 47.1 | 88 | +| SwinTransformer_large_patch4_window7_224[1] | 0.8596 | 0.9783 | 0.863 | 0.979 | 34.5 | 197 | +| SwinTransformer_large_patch4_window12_384[1] | 0.8719 | 0.9823 | 0.873 | 0.982 | 103.9 | 197 | + +[1]：基于 ImageNet22k 数据集预训练，然后在 ImageNet1k 数据集迁移学习得到。 + +**注**：与 Reference 的精度差异源于数据预处理不同。 + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| ------------------------------------------------------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| SwinTransformer_tiny_patch4_window7_224 | 224 | 256 | 6.59 | 9.68 | 16.32 | +| SwinTransformer_small_patch4_window7_224 | 224 | 256 | 12.54 | 17.07 | 28.08 | +| SwinTransformer_base_patch4_window7_224 | 224 | 256 | 13.37 | 23.53 | 39.11 | +| SwinTransformer_base_patch4_window12_384 | 384 | 384 | 19.52 | 64.56 | 123.30 | +| SwinTransformer_base_patch4_window7_224[1] | 224 | 256 | 13.53 | 23.46 | 39.13 | +| SwinTransformer_base_patch4_window12_384[1] | 384 | 384 | 19.65 | 64.72 | 123.42 | +| SwinTransformer_large_patch4_window7_224[1] | 224 | 256 | 15.74 | 38.57 | 71.49 | +| SwinTransformer_large_patch4_window12_384[1] | 384 | 384 | 32.61 | 116.59 | 223.23 | + +[1]：基于 ImageNet22k 数据集预训练，然后在 ImageNet1k 数据集迁移学习得到。 diff --git a/src/PaddleClas/docs/zh_CN/models/TNT.md b/src/PaddleClas/docs/zh_CN/models/TNT.md new file mode 100644 index 0000000..bf5181a --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/TNT.md @@ -0,0 +1,20 @@ +# TNT 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) + + + +## 1. 概述 + +TNT(Transformer-iN-Transformer)系列模型由华为诺亚于 2021 年提出，用于对 patch 级别和 pixel 级别的表示进行建模。在每个 TNT 块中，outer transformer block 用于处理 patch 嵌入，inner transformer block 从 pixel 嵌入中提取局部特征。通过线性变换层将 pixel 级特征投影到 patch 嵌入空间，然后加入到 patch 中。通过对 TNT 块的叠加，建立了用于图像识别的 TNT 模型。在 ImageNet 基准测试和下游任务上的实验证明了该 TNT 体系结构的优越性和有效性。例如，在计算量相当的情况下 TNT 能在 ImageNet 上达到 81.3% 的 top-1 精度，比 DeiT 高 1.5%。[论文地址](https://arxiv.org/abs/2103.00112)。 + + + +## 2. 精度、FLOPS 和参数量 + +| Model | Params (M) | FLOPs (G) | Top-1 (%) | Top-5 (%) | +|:---------------------:|:----------:|:---------:|:---------:|:---------:| +| TNT_small | 23.8 | 5.2 | 81.21 | 95.63 | diff --git a/src/PaddleClas/docs/zh_CN/models/Twins.md b/src/PaddleClas/docs/zh_CN/models/Twins.md new file mode 100644 index 0000000..623ebf8 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/Twins.md @@ -0,0 +1,40 @@ +# Twins +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于V100 GPU 的预测速度](#3) + + + +## 1. 概述 +Twins 网络包括 Twins-PCPVT 和 Twins-SVT，其重点对空间注意力机制进行了精心设计，得到了简单却更为有效的方案。由于该体系结构仅涉及矩阵乘法，而目前的深度学习框架中对矩阵乘法有较高的优化程度，因此该体系结构十分高效且易于实现。并且，该体系结构在图像分类、目标检测和语义分割等多种下游视觉任务中都能够取得优异的性能。[论文地址](https://arxiv.org/abs/2104.13840)。 + + + +## 2. 精度、FLOPs 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPs
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| pcpvt_small | 0.8082 | 0.9552 | 0.812 | - | 3.7 | 24.1 | +| pcpvt_base | 0.8242 | 0.9619 | 0.827 | - | 6.4 | 43.8 | +| pcpvt_large | 0.8273 | 0.9650 | 0.831 | - | 9.5 | 60.9 | +| alt_gvt_small | 0.8140 | 0.9546 | 0.817 | - | 2.8 | 24 | +| alt_gvt_base | 0.8294 | 0.9621 | 0.832 | - | 8.3 | 56 | +| alt_gvt_large | 0.8331 | 0.9642 | 0.837 | - | 14.8 | 99.2 | + +**注**：与 Reference 的精度差异源于数据预处理不同。 + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| ------------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| pcpvt_small | 224 | 256 | 7.32 | 10.51 | 15.27 | +| pcpvt_base | 224 | 256 | 12.20 | 16.22 | 23.16 | +| pcpvt_large | 224 | 256 | 16.47 | 22.90 | 32.73 | +| alt_gvt_small | 224 | 256 | 6.94 | 9.01 | 12.27 | +| alt_gvt_base | 224 | 256 | 9.37 | 15.02 | 24.54 | +| alt_gvt_large | 224 | 256 | 11.76 | 22.08 | 35.12 | diff --git a/src/PaddleClas/docs/zh_CN/models/ViT_and_DeiT.md b/src/PaddleClas/docs/zh_CN/models/ViT_and_DeiT.md new file mode 100644 index 0000000..51df939 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/ViT_and_DeiT.md @@ -0,0 +1,70 @@ +# ViT 与 DeiT 系列 +--- +## 目录 + +* [1. 概述](#1) +* [2. 精度、FLOPS 和参数量](#2) +* [3. 基于V100 GPU 的预测速度](#3) + + + +## 1. 概述 + +ViT（Vision Transformer）系列模型是 Google 在 2020 年提出的，该模型仅使用标准的 Transformer 结构，完全抛弃了卷积结构，将图像拆分为多个 patch 后再输入到 Transformer 中，展示了 Transformer 在 CV 领域的潜力。[论文地址](https://arxiv.org/abs/2010.11929)。 + +DeiT（Data-efficient Image Transformers）系列模型是由 FaceBook 在 2020 年底提出的，针对 ViT 模型需要大规模数据集训练的问题进行了改进，最终在 ImageNet 上取得了 83.1%的 Top1 精度。并且使用卷积模型作为教师模型，针对该模型进行知识蒸馏，在 ImageNet 数据集上可以达到 85.2% 的 Top1 精度。[论文地址](https://arxiv.org/abs/2012.12877)。 + + + + +## 2. 精度、FLOPS 和参数量 + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| ViT_small_patch16_224 | 0.7769 | 0.9342 | 0.7785 | 0.9342 | 9.41 | 48.60 | +| ViT_base_patch16_224 | 0.8195 | 0.9617 | 0.8178 | 0.9613 | 16.85 | 86.42 | +| ViT_base_patch16_384 | 0.8414 | 0.9717 | 0.8420 | 0.9722 | 49.35 | 86.42 | +| ViT_base_patch32_384 | 0.8176 | 0.9613 | 0.8166 | 0.9613 | 12.66 | 88.19 | +| ViT_large_patch16_224 | 0.8323 | 0.9650 | 0.8306 | 0.9644 | 59.65 | 304.12 | +| ViT_large_patch16_384 | 0.8513 | 0.9736 | 0.8517 | 0.9736 | 174.70 | 304.12 | +| ViT_large_patch32_384 | 0.8153 | 0.9608 | 0.815 | - | 44.24 | 306.48 | + + +| Models | Top1 | Top5 | Reference
top1 | Reference
top5 | FLOPS
(G) | Params
(M) | +|:--:|:--:|:--:|:--:|:--:|:--:|:--:| +| DeiT_tiny_patch16_224 | 0.718 | 0.910 | 0.722 | 0.911 | 1.07 | 5.68 | +| DeiT_small_patch16_224 | 0.796 | 0.949 | 0.799 | 0.950 | 4.24 | 21.97 | +| DeiT_base_patch16_224 | 0.817 | 0.957 | 0.818 | 0.956 | 16.85 | 86.42 | +| DeiT_base_patch16_384 | 0.830 | 0.962 | 0.829 | 0.972 | 49.35 | 86.42 | +| DeiT_tiny_distilled_patch16_224 | 0.741 | 0.918 | 0.745 | 0.919 | 1.08 | 5.87 | +| DeiT_small_distilled_patch16_224 | 0.809 | 0.953 | 0.812 | 0.954 | 4.26 | 22.36 | +| DeiT_base_distilled_patch16_224 | 0.831 | 0.964 | 0.834 | 0.965 | 16.93 | 87.18 | +| DeiT_base_distilled_patch16_384 | 0.851 | 0.973 | 0.852 | 0.972 | 49.43 | 87.18 | + +关于 Params、FLOPs、Inference speed 等信息，敬请期待。 + + + +## 3. 基于 V100 GPU 的预测速度 + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| -------------------------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| ViT_small_
patch16_224 | 256 | 224 | 3.71 | 9.05 | 16.72 | +| ViT_base_
patch16_224 | 256 | 224 | 6.12 | 14.84 | 28.51 | +| ViT_base_
patch16_384 | 384 | 384 | 14.15 | 48.38 | 95.06 | +| ViT_base_
patch32_384 | 384 | 384 | 4.94 | 13.43 | 24.08 | +| ViT_large_
patch16_224 | 256 | 224 | 15.53 | 49.50 | 94.09 | +| ViT_large_
patch16_384 | 384 | 384 | 39.51 | 152.46 | 304.06 | +| ViT_large_
patch32_384 | 384 | 384 | 11.44 | 36.09 | 70.63 | + +| Models | Crop Size | Resize Short Size | FP32
Batch Size=1
(ms) | FP32
Batch Size=4
(ms) | FP32
Batch Size=8
(ms) | +| ------------------------------------ | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ | +| DeiT_tiny_
patch16_224 | 256 | 224 | 3.61 | 3.94 | 6.10 | +| DeiT_small_
patch16_224 | 256 | 224 | 3.61 | 6.24 | 10.49 | +| DeiT_base_
patch16_224 | 256 | 224 | 6.13 | 14.87 | 28.50 | +| DeiT_base_
patch16_384 | 384 | 384 | 14.12 | 48.80 | 97.60 | +| DeiT_tiny_
distilled_patch16_224 | 256 | 224 | 3.51 | 4.05 | 6.03 | +| DeiT_small_
distilled_patch16_224 | 256 | 224 | 3.70 | 6.20 | 10.53 | +| DeiT_base_
distilled_patch16_224 | 256 | 224 | 6.17 | 14.94 | 28.58 | +| DeiT_base_
distilled_patch16_384 | 384 | 384 | 14.12 | 48.76 | 97.09 | + diff --git a/src/PaddleClas/docs/zh_CN/models/index.rst b/src/PaddleClas/docs/zh_CN/models/index.rst new file mode 100644 index 0000000..7873e3a --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models/index.rst @@ -0,0 +1,29 @@ +模型库 +================================ + +.. toctree:: + :maxdepth: 2 + + DPN_DenseNet.md + models_intro.md + RepVGG.md + EfficientNet_and_ResNeXt101_wsl.md + ViT_and_DeiT.md + SwinTransformer.md + Others.md + SEResNext_and_Res2Net.md + ESNet.md + HRNet.md + ReXNet.md + Inception.md + TNT.md + RedNet.md + DLA.md + ResNeSt_RegNet.md + PP-LCNet.md + HarDNet.md + ResNet_and_vd.md + LeViT.md + Mobile.md + MixNet.md + Twins.md diff --git a/src/PaddleClas/docs/zh_CN/models_training/classification.md b/src/PaddleClas/docs/zh_CN/models_training/classification.md new file mode 100644 index 0000000..dd95015 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models_training/classification.md @@ -0,0 +1,345 @@ +# 图像分类 +--- +图像分类是根据图像的语义信息将不同类别图像区分开来，是计算机视觉中重要的基本问题，也是图像检测、图像分割、物体跟踪、行为分析等其他高层视觉任务的基础。图像分类在很多领域有广泛应用，包括安防领域的人脸识别和智能视频分析等，交通领域的交通场景识别，互联网领域基于内容的图像检索和相册自动归类，医学领域的图像识别等。 + +一般来说，图像分类通过手工特征或特征学习方法对整个图像进行全部描述，然后使用分类器判别物体类别，因此如何提取图像的特征至关重要。在深度学习算法之前使用较多的是基于词袋(Bag of Words)模型的物体分类方法。而基于深度学习的图像分类方法，可以通过有监督或无监督的方式学习层次化的特征描述，从而取代了手工设计或选择图像特征的工作。深度学习模型中的卷积神经网络(Convolution Neural Network, CNN)近年来在图像领域取得了惊人的成绩，CNN 直接利用图像像素信息作为输入，最大程度上保留了输入图像的所有信息，通过卷积操作进行特征的提取和高层抽象，模型输出直接是图像识别的结果。这种基于“输入-输出”直接端到端的学习方法取得了非常好的效果，得到了广泛的应用。 + +图像分类是计算机视觉里很基础但又重要的一个领域，其研究成果一直影响着计算机视觉甚至深度学习的发展，图像分类有很多子领域，如多标签分类、细粒度分类等，此处只对单标签图像分类做一个简述。 + +具体图像分类算法介绍详见[文档](../algorithm_introduction/image_classification.md)。 + +## 目录 + +- [1. 数据集介绍](#1) + - [1.1 ImageNet-1k](#1.1) + - [1.2 CIFAR-10/CIFAR-100](#1.2) +- [2. 图像分类的流程](#2) + - [2.1 数据及其预处理](#2.1) + - [2.2 模型准备](#2.2) + - [2.3 模型训练](#2.3) + - [2.4 模型评估](#2.4) +- [3. 使用方法介绍](#3) + - [3.1 基于 CPU /单卡 GPU 上的训练与评估](#3.1) + - [3.1.1 模型训练](#3.1.1) + - [3.1.2 模型微调](#3.1.2) + - [3.1.3 模型恢复训练](#3.1.3) + - [3.1.4 模型评估](#3.1.4) + - [3.2 基于 Linux + 多卡 GPU 的模型训练与评估](#3.2) + - [3.2.1 模型训练](#3.2.1) + - [3.2.2 模型微调](#3.2.2) + - [3.2.3 模型恢复训练](#3.2.3) + - [3.2.4 模型评估](#3.2.4) + - [3.3 使用预训练模型进行模型预测](#3.3) + - [3.4 使用 inference 模型进行模型推理](#3.4) + + + +## 1. 数据集介绍 + + +### 1.1 ImageNet-1k + +ImageNet 项目是一个大型视觉数据库，用于视觉目标识别软件研究。该项目已手动注释了 1400 多万张图像，以指出图片中的对象，并在至少 100 万张图像中提供了边框。ImageNet-1k 是 ImageNet 数据集的子集，其包含 1000 个类别。训练集包含 1281167 个图像数据，验证集包含 50000 个图像数据。2010 年以来，ImageNet 项目每年举办一次图像分类竞赛，即 ImageNet 大规模视觉识别挑战赛(ILSVRC)。挑战赛使用的数据集即为 ImageNet-1k。到目前为止，ImageNet-1k 已经成为计算机视觉领域发展的最重要的数据集之一，其促进了整个计算机视觉的发展，很多计算机视觉下游任务的初始化模型都是基于该数据集训练得到的权重。 + + +### 1.2 CIFAR-10/CIFAR-100 + +CIFAR-10 数据集由 10 个类的 60000 个彩色图像组成，图像分辨率为 32x32，每个类有 6000 个图像，其中训练集 5000 张，验证集 1000 张，10 个不同的类代表飞机、汽车、鸟类、猫、鹿、狗、青蛙、马、轮船和卡车。CIFAR-100 数据集是 CIFAR-10 的扩展，由 100 个类的 60000 个彩色图像组成，图像分辨率为 32x32，每个类有 600 个图像，其中训练集 500 张，验证集 100 张。由于这两个数据集规模较小，因此可以让研究人员快速尝试不同的算法。这两个数据集也是图像分类领域测试模型好坏的常用数据集。 + + +## 2. 图像分类的流程 + +将准备好的训练数据做相应的数据预处理后经过图像分类模型，模型的输出与真实标签做交叉熵损失函数，该损失函数描述了模型的收敛方向，遍历所有的图片数据输入模型，对最终损失函数通过某些优化器做相应的梯度下降，将梯度信息回传到模型中，更新模型的权重，如此循环往复遍历多次数据，即可得到一个图像分类的模型。 + + +### 2.1 数据及其预处理 + +数据的质量及数量往往可以决定一个模型的好坏。在图像分类领域，数据包括图像及标签。在大部分情形下，带有标签的数据比较匮乏，所以数量很难达到使模型饱和的程度，为了可以使模型学习更多的图像特征，图像数据在进入模型之前要经过很多图像变换或者数据增强，来保证输入图像数据的多样性，从而保证模型有更好的泛化能力。PaddleClas 提供了训练 ImageNet-1k 的标准图像变换，也提供了多种数据增强的方法，相关代码可以查看[数据处理](../../../ppcls/data/preprocess)，配置文件可以参考[数据增强配置文件](../../../ppcls/configs/ImageNet/DataAugment)，相关数据增强算法详见[增强介绍文档](../algorithm_introduction/DataAugmentation.md)。 + + + +### 2.2 模型准备 + +在数据确定后，模型往往决定了最终算法精度的上限，在图像分类领域，经典的模型层出不穷，PaddleClas 提供了 35 个系列共 164 个 ImageNet 预训练模型。具体的精度、速度等指标请参考[骨干网络和预训练模型库](../algorithm_introduction/ImageNet_models.md)。 + + +### 2.3 模型训练 + +在准备好数据、模型后，便可以开始迭代模型并更新模型的参数。经过多次迭代最终可以得到训练好的模型来做图像分类任务。图像分类的训练过程需要很多经验，涉及很多超参数的设置，PaddleClas 提供了一些列的[训练调优方法](./train_strategy.md)，可以快速助你获得高精度的模型。 + +同时，PaddleClas 还支持使用VisualDL 可视化训练过程。VisualDL 是飞桨可视化分析工具，以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构，进而实现高效的模型优化。更多细节请查看[VisualDL](../others/VisualDL.md)。 + + +### 2.4 模型评估 + +当训练得到一个模型之后，如何确定模型的好坏，需要将模型在验证集上进行评估。评估指标一般是 Top1-Acc 或者 Top5-Acc，该指标越高往往代表模型性能越好。 + + +## 3. 使用方法介绍 + +请参考[安装指南](../installation/install_paddleclas.md)配置运行环境，并根据[快速开始](../quick_start/quick_start_classification_new_user.md)文档准备 flower102 数据集，本章节下面所有的实验均以 flower102 数据集为例。 + +PaddleClas 目前支持的训练/评估环境如下： +```shell +└── CPU/单卡 GPU +    ├── Linux +    └── Windows + +└── 多卡 GPU + └── Linux +``` + + +### 3.1 基于 CPU/单卡 GPU 上的训练与评估 + +在基于 CPU/单卡 GPU 上训练与评估，推荐使用 `tools/train.py` 与 `tools/eval.py` 脚本。关于 Linux 平台多卡 GPU 环境下的训练与评估，请参考 [3.2. 基于 Linux+GPU 的模型训练与评估](#3.2)。 + + + +#### 3.1.1 模型训练 + +准备好配置文件之后，可以使用下面的方式启动训练。 + +``` +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Arch.pretrained=False \ + -o Global.device=gpu +``` + +其中，`-c` 用于指定配置文件的路径，`-o` 用于指定需要修改或者添加的参数，其中 `-o Arch.pretrained=False` 表示不使用预训练模型，`-o Global.device=gpu` 表示使用 GPU 进行训练。如果希望使用 CPU 进行训练，则需要将 `Global.device` 设置为 `cpu`。 + +更详细的训练配置，也可以直接修改模型对应的配置文件。具体配置参数参考[配置文档](config_description.md)。 + +运行上述命令，可以看到输出日志，示例如下： + +* 如果在训练中使用了 mixup 或者 cutmix 的数据增广方式，那么日志中将不会打印 top-1 与 top-k（默认为 5）信息： + ``` + ... + [Train][Epoch 3/20][Avg]CELoss: 6.46287, loss: 6.46287 + ... + [Eval][Epoch 3][Avg]CELoss: 5.94309, loss: 5.94309, top1: 0.01961, top5: 0.07941 + ... + ``` + +* 如果训练过程中没有使用 mixup 或者 cutmix 的数据增广，那么除了上述信息外，日志中也会打印出 top-1 与 top-k（默认为 5）的信息： + + ``` + ... + [Train][Epoch 3/20][Avg]CELoss: 6.12570, loss: 6.12570, top1: 0.01765, top5: 0.06961 + ... + [Eval][Epoch 3][Avg]CELoss: 5.40727, loss: 5.40727, top1: 0.07549, top5: 0.20980 + ... + ``` + +训练期间也可以通过 VisualDL 实时观察 loss 变化，详见 [VisualDL](../others/VisualDL.md)。 + + +#### 3.1.2 模型微调 + +根据自己的数据集路径设置好配置文件后，可以通过加载预训练模型的方式进行微调，如下所示。 + +``` +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Arch.pretrained=True \ + -o Global.device=gpu +``` + +其中 `Arch.pretrained` 设置为 `True` 表示加载 ImageNet 的预训练模型，此外，`Arch.pretrained` 也可以指定具体的模型权重文件的地址，使用时需要换成自己的预训练模型权重文件的路径。 + +我们也提供了大量基于 `ImageNet-1k` 数据集的预训练模型，模型列表及下载地址详见[模型库概览](../algorithm_introduction/ImageNet_models.md)。 + + + +#### 3.1.3 模型恢复训练 + +如果训练任务因为其他原因被终止，也可以加载断点权重文件，继续训练： + +``` +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.checkpoints="./output/MobileNetV3_large_x1_0/epoch_5" \ + -o Global.device=gpu +``` + +其中配置文件不需要做任何修改，只需要在继续训练时设置 `Global.checkpoints` 参数即可，表示加载的断点权重文件路径，使用该参数会同时加载保存的断点权重和学习率、优化器等信息。 + +**注意**： + +* `-o Global.checkpoints` 参数无需包含断点权重文件的后缀名，上述训练命令会在训练过程中生成如下所示的断点权重文件，若想从断点 `5` 继续训练，则 `Global.checkpoints` 参数只需设置为 `"../output/MobileNetV3_large_x1_0/epoch_5"`，PaddleClas 会自动补充后缀名。output 目录下的文件结构如下所示： + + ```shell + output + ├── MobileNetV3_large_x1_0 + │   ├── best_model.pdopt + │   ├── best_model.pdparams + │   ├── best_model.pdstates + │   ├── epoch_1.pdopt + │   ├── epoch_1.pdparams + │   ├── epoch_1.pdstates + . + . + . + ``` + + + +#### 3.1.4 模型评估 + +可以通过以下命令进行模型评估。 + +```bash +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.pretrained_model=./output/MobileNetV3_large_x1_0/best_model +``` + +上述命令将使用 `./configs/quick_start/MobileNetV3_large_x1_0.yaml` 作为配置文件，对上述训练得到的模型 `./output/MobileNetV3_large_x1_0/best_model` 进行评估。你也可以通过更改配置文件中的参数来设置评估，也可以通过 `-o` 参数更新配置，如上所示。 + +可配置的部分评估参数说明如下： +* `Arch.name`：模型名称 +* `Global.pretrained_model`：待评估的模型预训练模型文件路径 + +**注意：** 在加载待评估模型时，需要指定模型文件的路径，但无需包含文件后缀名，PaddleClas 会自动补齐 `.pdparams` 的后缀，如 [3.1.3 模型恢复训练](#3.1.3)。 + + + +### 3.2 基于 Linux + 多卡 GPU 的模型训练与评估 + +如果机器环境为 Linux + GPU，那么推荐使用 `paddle.distributed.launch` 启动模型训练脚本(`tools/train.py`)、评估脚本(`tools/eval.py`)，可以更方便地启动多卡训练与评估。 + + + +#### 3.2.1 模型训练 + +参考如下方式启动模型训练，`paddle.distributed.launch` 通过设置 `gpus` 指定 GPU 运行卡号： + +```bash +# PaddleClas 通过 launch 方式启动多卡多进程训练 + +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml +``` + +输出日志信息的格式同上，详见 [3.1.1 模型训练](#3.1.1)。 + + +#### 3.2.2 模型微调 + +根据自己的数据集配置好配置文件之后，可以加载预训练模型进行微调，如下所示。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Arch.pretrained=True +``` + +其中 `Arch.pretrained` 为 `True` 或 `False`，当然也可以设置加载预训练权重文件的路径，使用时需要换成自己的预训练模型权重文件路径，也可以直接在配置文件中修改该路径。 + +30 分钟玩转 PaddleClas [尝鲜版](../quick_start/quick_start_classification_new_user.md)与[进阶版](../quick_start/quick_start_classification_professional.md)中包含大量模型微调的示例，可以参考该章节在特定的数据集上进行模型微调。 + + + +#### 3.2.3 模型恢复训练 + +如果训练任务因为其他原因被终止，也可以加载断点权重文件继续训练。 + +``` +export CUDA_VISIBLE_DEVICES=0,1,2,3 + +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.checkpoints="./output/MobileNetV3_large_x1_0/epoch_5" \ + -o Global.device=gpu +``` + +其中配置文件不需要做任何修改，只需要在训练时设置 `Global.checkpoints` 参数即可，该参数表示加载的断点权重文件路径，使用该参数会同时加载保存的模型参数权重和学习率、优化器等信息，详见 [3.1.3 模型恢复训练](#3.1.3)。 + + + +#### 3.2.4 模型评估 + +可以通过以下命令进行模型评估。 + +```bash +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.pretrained_model=./output/MobileNetV3_large_x1_0/best_model +``` + +参数说明详见 [3.1.4 模型评估](#3.1.4)。 + + + +### 3.3 使用预训练模型进行模型预测 + +模型训练完成之后，可以加载训练得到的预训练模型，进行模型预测。在模型库的 `tools/infer/infer.py` 中提供了完整的示例，只需执行下述命令即可完成模型预测： + +```python +python3 tools/infer.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Infer.infer_imgs=dataset/flowers102/jpg/image_00001.jpg \ + -o Global.pretrained_model=./output/MobileNetV3_large_x1_0/best_model +``` + +参数说明： ++ `Infer.infer_imgs`：待预测的图片文件路径或者批量预测时的图片文件夹。 ++ `Global.pretrained_model`：模型权重文件路径，如 `./output/MobileNetV3_large_x1_0/best_model` + + + +### 3.4 使用 inference 模型进行模型推理 + +通过导出 inference 模型，PaddlePaddle 支持使用预测引擎进行预测推理。接下来介绍如何用预测引擎进行推理： +首先，对训练好的模型进行转换： + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o Global.pretrained_model=output/MobileNetV3_large_x1_0/best_model +``` + + +其中，`Global.pretrained_model` 用于指定模型文件路径，该路径仍无需包含模型文件后缀名（如 [3.1.3 模型恢复训练](#3.1.3)）。 + +上述命令将生成模型结构文件(`inference.pdmodel`)和模型权重文件(`inference.pdiparams`)，然后可以使用预测引擎进行推理： + +进入 deploy 目录下： + +```bash +cd deploy +``` + +执行命令进行预测，由于默认 `class_id_map_file` 是 ImageNet 数据集的映射文件，所以此处需要置 None。 + +```bash +python3 python/predict_cls.py \ + -c configs/inference_cls.yaml \ + -o Global.infer_imgs=../dataset/flowers102/jpg/image_00001.jpg \ + -o Global.inference_model_dir=../inference/ \ + -o PostProcess.Topk.class_id_map_file=None + +``` +其中： ++ `Global.infer_imgs`：待预测的图片文件路径。 ++ `Global.inference_model_dir`：inference 模型结构文件路径，如 `../inference/inference.pdmodel` ++ `Global.use_tensorrt`：是否使用 TesorRT 预测引擎，默认值：`False` ++ `Global.use_gpu`：是否使用 GPU 预测，默认值：`True` ++ `Global.enable_mkldnn`：是否启用 `MKL-DNN` 加速，默认为 `False`。注意 `enable_mkldnn` 与 `use_gpu` 同时为 `True` 时，将忽略 `enable_mkldnn`，而使用 GPU 运行。 ++ `Global.use_fp16`：是否启用 `FP16`，默认为 `False`。 + + +注意: 如果使用 `Transformer` 系列模型，如 `DeiT_***_384`, `ViT_***_384` 等，请注意模型的输入数据尺寸，需要设置参数 `resize_short=384`, `resize=384`。 + +如果你希望提升评测模型速度，使用 GPU 评测时，建议开启 TensorRT 加速预测，使用 CPU 评测时，建议开启 MKLDNN 加速预测。 diff --git a/src/PaddleClas/docs/zh_CN/models_training/config_description.md b/src/PaddleClas/docs/zh_CN/models_training/config_description.md new file mode 100644 index 0000000..8c51d7a --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models_training/config_description.md @@ -0,0 +1,273 @@ +# 配置说明 + +--- + +## 简介 + +本文档介绍了 PaddleClas 配置文件(`ppcls/configs/*.yaml`)中各参数的含义，以便您更快地自定义或修改超参数配置。 + + + +## 配置详解 + +### 目录 + +- [1. 分类模型](#1) + - [1.1 全局配置(Global)](#1.1) + - [1.2 结构(Arch)](#1.2) + - [1.3 损失函数(Loss)](#1.3) + - [1.4 优化器(Optimizer)](#1.4) + - [1.5 数据读取模块(DataLoader)](#1.5) + - [1.5.1 dataset](#1.5.1) + - [1.5.2 sampler](#1.5.2) + - [1.5.3 loader](#1.5.3) + - [1.6 评估指标(Metric)](#1.6) + - [1.7 预测](#1.7) +- [2. 蒸馏模型](#2) + - [2.1 结构(Arch)](#2.1) + - [2.2 损失函数(Loss)](#2.2) + - [2.3 评估指标(Metric)](#2.3) +- [3. 识别模型](#3) + - [3.1 结构(Arch)](#3.1) + - [3.2 评估指标(Metric)](#3.2) + + + +### 1.分类模型 + +此处以 `ResNet50_vd` 在 `ImageNet-1k` 上的训练配置为例，详解各个参数的意义。[配置路径](../../../ppcls/configs/ImageNet/ResNet/ResNet50_vd.yaml)。 + + +#### 1.1 全局配置(Global) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| checkpoints | 断点模型路径，用于恢复训练 | null | str | +| pretrained_model | 预训练模型路径 | null | str | +| output_dir | 保存模型路径 | "./output/" | str | +| save_interval | 每隔多少个 epoch 保存模型 | 1 | int | +| eval_during_train| 是否在训练时进行评估 | True | bool | +| eval_interval | 每隔多少个 epoch 进行模型评估 | 1 | int | +| epochs | 训练总 epoch 数 | | int | +| print_batch_step | 每隔多少个 mini-batch 打印输出 | 10 | int | +| use_visualdl | 是否是用 visualdl 可视化训练过程 | False | bool | +| image_shape | 图片大小 | [3, 224, 224] | list, shape: (3,) | +| save_inference_dir | inference 模型的保存路径 | "./inference" | str | +| eval_mode | eval 的模式 | "classification" | "retrieval" | +| to_static | 是否改为静态图模式 | False | True | +| ues_dali | 是否使用 dali 库进行图像预处理 | False | True | + +**注**：`pretrained_model` 也可以填写存放预训练模型的 http 地址。 + + +#### 1.2 结构(Arch) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| name | 模型结构名字 | ResNet50 | PaddleClas 提供的模型结构 | +| class_num | 分类数 | 1000 | int | +| pretrained | 预训练模型 | False | bool, str | + +**注**：此处的 pretrained 可以设置为 `True` 或者 `False`，也可以设置权重的路径。另外当 `Global.pretrained_model` 也设置相应路径时，此处的 `pretrained` 失效。 + + +#### 1.3 损失函数（Loss） + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| CELoss | 交叉熵损失函数 | —— | —— | +| CELoss.weight | CELoss 在整个 Loss 中的权重 | 1.0 | float | +| CELoss.epsilon | CELoss 中 label_smooth 的 epsilon 值 | 0.1 | float, 0-1 之间 | + + +#### 1.4 优化器(Optimizer) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| name | 优化器方法名 | "Momentum" | "RmsProp"等其他优化器 | +| momentum | momentum 值 | 0.9 | float | +| lr.name | 学习率下降方式 | "Cosine" | "Linear"、"Piecewise"等其他下降方式 | +| lr.learning_rate | 学习率初始值 | 0.1 | float | +| lr.warmup_epoch | warmup 轮数 | 0 | int，如 5 | +| regularizer.name | 正则化方法名 | "L2" | ["L1", "L2"] | +| regularizer.coeff | 正则化系数 | 0.00007 | float | + +**注**：`lr.name` 不同时，新增的参数可能也不同，如当 `lr.name=Piecewise` 时，需要添加如下参数： + +``` + lr: + name: Piecewise + learning_rate: 0.1 + decay_epochs: [30, 60, 90] + values: [0.1, 0.01, 0.001, 0.0001] +``` + +添加方法及参数请查看 [learning_rate.py](../../../ppcls/optimizer/learning_rate.py)。 + + +#### 1.5 数据读取模块(DataLoader) + + +##### 1.5.1 dataset + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| name | 读取数据的类的名字 | ImageNetDataset | VeriWild 等其他读取数据类的名字 | +| image_root | 数据集存放的路径 | ./dataset/ILSVRC2012/ | str | +| cls_label_path | 数据集标签 list | ./dataset/ILSVRC2012/train_list.txt | str | +| transform_ops | 单张图片的数据预处理 | —— | —— | +| batch_transform_ops | batch 图片的数据预处理 | —— | —— | + + +transform_ops 中参数的意义： + +| 功能名字 | 参数名字 | 具体含义 | +|:---:|:---:|:---:| +| DecodeImage | to_rgb | 数据转 RGB | +| | channel_first | 按 CHW 排列的图片数据 | +| RandCropImage | size | 随机裁剪 | +| RandFlipImage | | 随机翻转 | +| NormalizeImage | scale | 归一化 scale 值 | +| | mean | 归一化均值 | +| | std | 归一化方差 | +| | order | 归一化顺序 | +| CropImage | size | 裁剪大小 | +| ResizeImage | resize_short | 按短边调整大小 | + +batch_transform_ops 中参数的含义： + +| 功能名字 | 参数名字 | 具体含义 | +|:---:|:---:|:---:| +| MixupOperator | alpha | Mixup 参数值，该值越大增强越强 | + + +##### 1.5.2 sampler + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| name | sampler 类型 | DistributedBatchSampler | DistributedRandomIdentitySampler 等其他 Sampler | +| batch_size | 批大小 | 64 | int | +| drop_last | 是否丢掉最后不够 batch-size 的数据 | False | bool | +| shuffle | 数据是否做 shuffle | True | bool | + + +##### 1.5.3 loader + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| num_workers | 数据读取线程数 | 4 | int | +| use_shared_memory | 是否使用共享内存 | True | bool | + + +#### 1.6 评估指标(Metric) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| TopkAcc | TopkAcc | [1, 5] | list, int | + + +#### 1.7 预测（Infer） + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| infer_imgs | 被 infer 的图像的地址 | docs/images/whl/demo.jpg | str | +| batch_size | 批大小 | 10 | int | +| PostProcess.name | 后处理名字 | Topk | str | +| PostProcess.topk | topk 的值 | 5 | int | +| PostProcess.class_id_map_file | class id 和名字的映射文件 | ppcls/utils/imagenet1k_label_list.txt | str | + +**注**：Infer 模块的 `transforms` 的解释参考数据读取模块中的 dataset 中 `transform_ops` 的解释。 + + +### 2.蒸馏模型 + +**注**：此处以 `MobileNetV3_large_x1_0` 在 `ImageNet-1k` 上蒸馏 `MobileNetV3_small_x1_0` 的训练配置为例，详解各个参数的意义。[配置路径](../../../ppcls/configs/ImageNet/Distillation/mv3_large_x1_0_distill_mv3_small_x1_0.yaml)。这里只介绍与分类模型有区别的参数。 + + +#### 2.1 结构（Arch） + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| name | 模型结构名字 | DistillationModel | —— | +| class_num | 分类数 | 1000 | int | +| freeze_params_list | 冻结参数列表 | [True, False] | list | +| models | 模型列表 | [Teacher, Student] | list | +| Teacher.name | 教师模型的名字 | MobileNetV3_large_x1_0 | PaddleClas 中的模型 | +| Teacher.pretrained | 教师模型预训练权重 | True | 布尔值或者预训练权重路径 | +| Teacher.use_ssld | 教师模型预训练权重是否是 ssld 权重 | True | 布尔值 | +| infer_model_name | 被 infer 模型的类型 | Student | Teacher | + +**注**： + +1.list 在 yaml 中体现如下： + +``` + freeze_params_list: + - True + - False +``` +2.Student 的参数情况类似，不再赘述。 + + +#### 2.2 损失函数(Loss) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| DistillationCELoss | 蒸馏的交叉熵损失函数 | —— | —— | +| DistillationCELoss.weight | Loss 权重 | 1.0 | float | +| DistillationCELoss.model_name_pairs | ["Student", "Teacher"] | —— | —— | +| DistillationGTCELoss.weight | 蒸馏的模型与真实 Label 的交叉熵损失函数 | —— | —— | +| DistillationGTCELos.weight | Loss 权重 | 1.0 | float | +| DistillationCELoss.model_names | 与真实 label 作交叉熵的模型名字 | ["Student"] | —— | + + +#### 2.3 评估指标(Metric) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| DistillationTopkAcc | DistillationTopkAcc | 包含 model_key 和 topk 两个参数 | —— | +| DistillationTopkAcc.model_key | 被评估的模型 | "Student" | "Teacher" | +| DistillationTopkAcc.topk | Topk 的值 | [1, 5] | list, int | + +**注**：`DistillationTopkAcc` 与普通 `TopkAcc` 含义相同，只是只用在蒸馏任务中。 + + +### 3. 识别模型 + +**注**：此处以 `ResNet50` 在 `LogoDet-3k` 上的训练配置为例，详解各个参数的意义。[配置路径](../../../ppcls/configs/Logo/ResNet50_ReID.yaml)。这里只介绍与分类模型有区别的参数。 + + +#### 3.1 结构(Arch) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +| :---------------: | :-----------------------: | :--------: | :----------------------------------------------------------: | +| name | 模型结构 | "RecModel" | ["RecModel"] | +| infer_output_key | inference 时的输出值 | “feature” | ["feature", "logits"] | +| infer_add_softmax | infercne 是否添加 softmax | False | [True, False] | +| Backbone.name | Backbone 的名字 | ResNet50_last_stage_stride1 | PaddleClas 提供的其他 backbone | +| Backbone.pretrained | Backbone 预训练模型 | True | 布尔值或者预训练模型路径 | +| BackboneStopLayer.name | Backbone 中的输出层名字 | True | Backbone 中的特征输出层的 `full_name` | +| Neck.name | 网络 Neck 部分名字 | VehicleNeck | 需传入字典结构，Neck 网络层的具体输入参数 | +| Neck.in_channels | 输入 Neck 部分的维度大小 | 2048 | 与 BackboneStopLayer.name 层的大小相同 | +| Neck.out_channels | 输出 Neck 部分的维度大小，即特征维度大小 | 512 | int | +| Head.name | 网络 Head 部分名字 | CircleMargin | Arcmargin 等 | +| Head.embedding_size | 特征维度大小 | 512 | 与 Neck.out_channels 保持一致 | +| Head.class_num | 类别数 | 3000 | int | +| Head.margin | CircleMargin 中的 margin 值 | 0.35 | float | +| Head.scale | CircleMargin 中的 scale 值 | 64 | int | + +**注**： + +1.在 PaddleClas 中，`Neck` 部分是 Backbone 与 embedding 层的连接部分，`Head` 部分是 embedding 层与分类层的连接部分。 + +2.`BackboneStopLayer.name` 的获取方式可以通过将模型可视化后获取，可视化方式可以参考 [Netron](https://github.com/lutzroeder/netron) 或者 [visualdl](https://github.com/PaddlePaddle/VisualDL)。 + +3.调用 `tools/export_model.py` 会将模型的权重转为 inference model，其中 `infer_add_softmax` 参数会控制是否在其后增加 `Softmax` 激活函数，代码中默认为 `True`（分类任务中最后的输出层会接 `Softmax` 激活函数），识别任务中特征层无须接激活函数，此处要设置为 `False`。 + + +#### 3.2 评估指标(Metric) + +| 参数名字 | 具体含义 | 默认值 | 可选值 | +|:---:|:---:|:---:|:---:| +| Recallk| 召回率 | [1, 5] | list, int | +| mAP| 平均检索精度 | None | None | diff --git a/src/PaddleClas/docs/zh_CN/models_training/index.rst b/src/PaddleClas/docs/zh_CN/models_training/index.rst new file mode 100644 index 0000000..41ea472 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models_training/index.rst @@ -0,0 +1,10 @@ +模型训练 +================================ + +.. toctree:: + :maxdepth: 2 + + config_description.md + classification.md + recognition.md + train_strategy.md diff --git a/src/PaddleClas/docs/zh_CN/models_training/recognition.md b/src/PaddleClas/docs/zh_CN/models_training/recognition.md new file mode 100644 index 0000000..6f044d5 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models_training/recognition.md @@ -0,0 +1,337 @@ +# 图像识别 +--- +在 PaddleClas 中，图像识别，是指给定一张查询图像，系统能够识别该查询图像类别。广义上，图像分类也是图像识别的一种。但是与普通图像识别不同的是，图像分类只能判别出模型已经学习的类别，如果需要添加新的类别，分类模型只能重新训练。PaddleClas 中的图像识别，**对于陌生类别，只需要更新相应的检索库**，就能够正确的识别出查询图像的类别，而无需重新训练模型，这大大增加了识别系统的可用性，同时降低了更新模型的需求，方便用户部署应用。 + +对于一张待查询图片，PaddleClas 中的图像识别流程主要分为三部分： + +1. 主体检测：对于给定一个查询图像，主体检测器首先检测出图像的物体，从而去掉无用背景信息，提高识别精度。 +2. 特征提取：对主体检测的各个候选区域，通过特征模型，进行特征提取 +3. 特征检索：将提取的特征与特征库中的向量进行相似度比对，得到其标签信息 + +其中特征库，需要利用已经标注好的图像数据集提前建立。完整的图像识别系统，如下图所示 + +![](../../images/structure.jpg) +体验整体图像识别系统，或查看特征库建立方法，详见[图像识别快速开始文档](../quick_start/quick_start_recognition.md)。其中，图像识别快速开始文档主要讲解整体流程的使用过程。以下内容，主要对上述三个步骤的训练部分进行介绍。 + +首先，请参考[安装指南](../installation/install_paddleclas.md)配置运行环境。 + +## 目录 + +- [1. 主体检测](#1) +- [2. 特征模型训练](#2) + - [2.1. 特征模型数据准备与处理](#2.1) + - [2. 2 特征模型基于单卡 GPU 上的训练与评估](#2.2) + - [2.2.1 特征模型训练](#2.2.2) + - [2.2.2 特征模型恢复训练](#2.2.2) + - [2.2.3 特征模型评估](#2.2.3) + - [2.3 特征模型导出 inference 模型](#2.3) +- [3. 特征检索](#3) +- [4. 基础知识](#4) + + + +## 1. 主体检测 + +主体检测训练过程基于 [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection/tree/develop)，唯一的区别在于，主体检测任务中，所有的检测框均属于前景，在这里需要将标注文件中，检测框的 `category_id` 修改为 1，同时将整个标注文件中的 `categories` 映射表修改为下面的格式，即整个类别映射表中只包含 `前景` 类别。 + +```json +[{u'id': 1, u'name': u'foreground', u'supercategory': u'foreground'}] +``` + +关于主体检测训练方法可以参考： [PaddleDetection 训练教程](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md#4-%E8%AE%AD%E7%BB%83)。 + +更多关于 PaddleClas 中提供的主体检测的模型介绍与下载请参考：[主体检测教程](../image_recognition_pipeline/mainbody_detection.md)。 + + + +## 2. 特征模型训练 + + + +### 2.1 特征模型数据的准备与处理 + +* 进入 `PaddleClas` 目录。 + +```bash +## linux or mac， $path_to_PaddleClas 表示 PaddleClas 的根目录，用户需要根据自己的真实目录修改 +cd $path_to_PaddleClas +``` + +* 进入 `dataset` 目录，为了快速体验 PaddleClas 图像检索模块，此处使用的数据集为 [CUB_200_2011](http://vision.ucsd.edu/sites/default/files/WelinderEtal10_CUB-200.pdf)，其是一个包含 200 类鸟的细粒度鸟类数据集。首先，下载 CUB_200_2011 数据集，下载方式请参考[官网](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html)。 + +```shell +# linux or mac +cd dataset + +# 将下载后的数据拷贝到此目录 +cp {数据存放的路径}/CUB_200_2011.tgz . + +# 解压 +tar -xzvf CUB_200_2011.tgz + +#进入 CUB_200_2011 目录 +cd CUB_200_2011 +``` + +该数据集在用作图像检索任务时，通常将前 100 类当做训练集，后 100 类当做测试集，所以此处需要将下载的数据集做一些后处理，来更好的适应 PaddleClas 的图像检索训练。 + +```shell +#新建 train 和 test 目录 +mkdir train && mkdir test + +#将数据分成训练集和测试集，前 100 类作为训练集，后 100 类作为测试集 +ls images | awk -F "." '{if(int($1)<101)print "mv images/"$0" train/"int($1)}' | sh +ls images | awk -F "." '{if(int($1)>100)print "mv images/"$0" test/"int($1)}' | sh + +#生成 train_list 和 test_list +tree -r -i -f train | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > train_list.txt +tree -r -i -f test | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > test_list.txt +``` + +至此，现在已经得到 `CUB_200_2011` 的训练集（`train` 目录）、测试集（`test` 目录）、`train_list.txt`、`test_list.txt`。 + +数据处理完毕后，`CUB_200_2011` 中的 `train` 目录下应有如下结构： + +``` +├── 1 +│ ├── Black_Footed_Albatross_0001_796111.jpg +│ ├── Black_Footed_Albatross_0002_55.jpg + ... +├── 10 +│ ├── Red_Winged_Blackbird_0001_3695.jpg +│ ├── Red_Winged_Blackbird_0005_5636.jpg +... +``` + +`train_list.txt` 应为： + +``` +train/99/Ovenbird_0137_92639.jpg 99 1 +train/99/Ovenbird_0136_92859.jpg 99 2 +train/99/Ovenbird_0135_93168.jpg 99 3 +train/99/Ovenbird_0131_92559.jpg 99 4 +train/99/Ovenbird_0130_92452.jpg 99 5 +... +``` +其中，分隔符为空格" ", 三列数据的含义分别是训练数据的路径、训练数据的 label 信息、训练数据的 unique id。 + +测试集格式与训练集格式相同。 + +**注意**： + +* 当 gallery dataset 和 query dataset 相同时，为了去掉检索得到的第一个数据（检索图片本身无须评估），每个数据需要对应一个 unique id，用于后续评测 mAP、recall@1 等指标。关于 gallery dataset 与 query dataset 的解析请参考[图像检索数据集介绍](#图像检索数据集介绍), 关于 mAP、recall@1 等评测指标请参考[图像检索评价指标](#图像检索评价指标)。 + +返回 `PaddleClas` 根目录 + +```shell +# linux or mac +cd ../../ +``` + + + +### 2.2 特征模型 GPU 上的训练与评估 + +在基于单卡 GPU 上训练与评估，推荐使用 `tools/train.py` 与 `tools/eval.py` 脚本。 + +PaddleClas 支持使用 VisualDL 可视化训练过程。VisualDL 是飞桨可视化分析工具，以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构，进而实现高效的模型优化。更多细节请查看[VisualDL](../others/VisualDL.md)。 + + + +#### 2.2.1 特征模型训练 + +准备好配置文件之后，可以使用下面的方式启动图像检索任务的训练。PaddleClas 训练图像检索任务的方法是度量学习，关于度量学习的解析请参考[度量学习](#度量学习)。 + +```shell +# 单卡 GPU +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Arch.Backbone.pretrained=True \ + -o Global.device=gpu +# 多卡 GPU +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Arch.Backbone.pretrained=True \ + -o Global.device=gpu +``` + +其中，`-c` 用于指定配置文件的路径，`-o` 用于指定需要修改或者添加的参数，其中 `-o Arch.Backbone.pretrained=True` 表示 Backbone 部分使用预训练模型，此外，`Arch.Backbone.pretrained` 也可以指定具体的模型权重文件的地址，使用时需要换成自己的预训练模型权重文件的路径。`-o Global.device=gpu` 表示使用 GPU 进行训练。如果希望使用 CPU 进行训练，则需要将 `Global.device` 设置为 `cpu`。 + +更详细的训练配置，也可以直接修改模型对应的配置文件。具体配置参数参考[配置文档](config_description.md)。 + +运行上述命令，可以看到输出日志，示例如下： + + ``` + ... + [Train][Epoch 1/50][Avg]CELoss: 6.59110, TripletLossV2: 0.54044, loss: 7.13154 + ... + [Eval][Epoch 1][Avg]recall1: 0.46962, recall5: 0.75608, mAP: 0.21238 + ... + ``` +此处配置文件的 Backbone 是 MobileNetV1，如果想使用其他 Backbone，可以重写参数 `Arch.Backbone.name`，比如命令中增加 `-o Arch.Backbone.name={其他 Backbone}`。此外，由于不同模型 `Neck` 部分的输入维度不同，更换 Backbone 后可能需要改写此处的输入大小，改写方式类似替换 Backbone 的名字。 + +在训练 Loss 部分，此处使用了 [CELoss](../../../ppcls/loss/celoss.py) 和 [TripletLossV2](../../../ppcls/loss/triplet.py)，配置文件如下： + +``` +Loss: + Train: + - CELoss: + weight: 1.0 + - TripletLossV2: + weight: 1.0 + margin: 0.5 +``` + +最终的总 Loss 是所有 Loss 的加权和，其中 weight 定义了特定 Loss 在最终总 Loss 的权重。如果想替换其他 Loss，也可以在配置文件中更改 Loss 字段，目前支持的 Loss 请参考 [Loss](../../../ppcls/loss)。 + + + +#### 2.2.2 特征模型恢复训练 + +如果训练任务因为其他原因被终止，也可以加载断点权重文件，继续训练： + +```shell +# 单卡 +python3 tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.checkpoints="./output/RecModel/epoch_5" \ + -o Global.device=gpu +# 多卡 +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch tools/train.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.checkpoints="./output/RecModel/epoch_5" \ + -o Global.device=gpu +``` + +其中配置文件不需要做任何修改，只需要在继续训练时设置 `Global.checkpoints` 参数即可，表示加载的断点权重文件路径，使用该参数会同时加载保存的断点权重和学习率、优化器等信息。 + +**注意**： + +* `-o Global.checkpoints` 参数无需包含断点权重文件的后缀名，上述训练命令会在训练过程中生成如下所示的断点权重文件，若想从断点 `5` 继续训练，则 `Global.checkpoints` 参数只需设置为 `"./output/RecModel/epoch_5"`，PaddleClas 会自动补充后缀名。 + + ```shell + output/ + └── RecModel + ├── best_model.pdopt + ├── best_model.pdparams + ├── best_model.pdstates + ├── epoch_1.pdopt + ├── epoch_1.pdparams + ├── epoch_1.pdstates + . + . + . + ``` + + + +#### 2.2.3 特征模型评估 + +可以通过以下命令进行模型评估。 + +```bash +# 单卡 +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.pretrained_model=./output/RecModel/best_model +# 多卡 +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch tools/eval.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.pretrained_model=./output/RecModel/best_model +``` + +上述命令将使用 `./configs/quick_start/MobileNetV1_retrieval.yaml` 作为配置文件，对上述训练得到的模型 `./output/RecModel/best_model` 进行评估。你也可以通过更改配置文件中的参数来设置评估，也可以通过 `-o` 参数更新配置，如上所示。 + +可配置的部分评估参数说明如下： +* `Arch.name`：模型名称 +* `Global.pretrained_model`：待评估的模型的预训练模型文件路径，不同于 `Global.Backbone.pretrained`，此处的预训练模型是整个模型的权重，而 `Global.Backbone.pretrained` 只是 Backbone 部分的权重。当需要做模型评估时，需要加载整个模型的权重。 +* `Metric.Eval`：待评估的指标，默认评估 recall@1、recall@5、mAP。当你不准备评测某一项指标时，可以将对应的试标从配置文件中删除；当你想增加某一项评测指标时，也可以参考 [Metric](../../../ppcls/metric/metrics.py) 部分在配置文件 `Metric.Eval` 中添加相关的指标。 + +**注意：** + +* 在加载待评估模型时，需要指定模型文件的路径，但无需包含文件后缀名，PaddleClas 会自动补齐 `.pdparams` 的后缀，如 [2.2.2 特征模型恢复训练](#2.2.2)。 + +* Metric learning 任务一般不评测 TopkAcc。 + + + +### 2.3 特征模型导出 inference 模型 + +通过导出 inference 模型，PaddlePaddle 支持使用预测引擎进行预测推理。对训练好的模型进行转换： + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \ + -o Global.pretrained_model=output/RecModel/best_model \ + -o Global.save_inference_dir=./inference +``` + +其中，`Global.pretrained_model` 用于指定模型文件路径，该路径仍无需包含模型文件后缀名（如[2.2.2 特征模型恢复训练](#2.2.2)）。当执行后，会在当前目录下生成 `./inference` 目录，目录下包含 `inference.pdiparams`、`inference.pdiparams.info`、`inference.pdmodel` 文件。`Global.save_inference_dir` 可以指定导出 inference 模型的路径。此处保存的 inference 模型在 embedding 特征层做了截断，即模型最终的输出为 n 维 embedding 特征。 + +上述命令将生成模型结构文件(`inference.pdmodel`)和模型权重文件(`inference.pdiparams`)，然后可以使用预测引擎进行推理。使用 inference 模型推理的流程可以参考[基于 Python 预测引擎预测推理](../inference_deployment/python_deploy.md)。 + + + +## 3. 特征检索 + +PaddleClas 图像检索部分目前支持的环境如下： + +```shell +└── CPU/单卡 GPU + ├── Linux + ├── MacOS + └── Windows +``` + +此部分使用了 [Faiss](https://github.com/facebookresearch/faiss) 作为检索库，其是一个高效的特征检索及聚类的库。此库中集成了多种相似度检索算法，以满足不同的检索场景。在 PaddleClas 中，支持三种检索算法： + +- **HNSW32**: 一种图索引方法。检索精度较高，速度较快。但是特征库只支持添加图像功能，不支持删除图像特征功能。（默认方法） +- **IVF**：倒排索引检索方法。速度较快，但是精度略低。特征库支持增加、删除图像特功能。 +- **FLAT**： 暴力检索算法。精度最高，但是数据量大时，检索速度较慢。特征库支持增加、删除图像特征功能。 + +详细介绍请参考 [Faiss](https://github.com/facebookresearch/faiss) 官方文档。 + +具体安装方法如下： + +```python +pip install faiss-cpu==1.7.1post2 +``` + +若使用时，不能正常引用，则 `uninstall` 之后，重新 `install`，尤其是 `windows` 下。 + + + +## 4. 基础知识 + +图像检索指的是给定一个包含特定实例(例如特定目标、场景、物品等)的查询图像，图像检索旨在从数据库图像中找到包含相同实例的图像。不同于图像分类，图像检索解决的是一个开集问题，训练集中可能不包含被识别的图像的类别。图像检索的整体流程为：首先将图像中表示为一个合适的特征向量，其次，对这些图像的特征向量用欧式距离或余弦距离进行最近邻搜索以找到底库中相似的图像，最后，可以使用一些后处理技术对检索结果进行微调，确定被识别图像的类别等信息。所以，决定一个图像检索算法性能的关键在于图像对应的特征向量的好坏。 + + +- 度量学习（Metric Learning） + +度量学习研究如何在一个特定的任务上学习一个距离函数，使得该距离函数能够帮助基于近邻的算法（kNN、k-means 等）取得较好的性能。深度度量学习(Deep Metric Learning)是度量学习的一种方法，它的目标是学习一个从原始特征到低维稠密的向量空间（嵌入空间，embedding space）的映射，使得同类对象在嵌入空间上使用常用的距离函数（欧氏距离、cosine 距离等）计算的距离比较近，而不同类的对象之间的距离则比较远。深度度量学习在计算机视觉领域取得了非常多的成功的应用，比如人脸识别、商品识别、图像检索、行人重识别等。更详细的介绍请参考[此文档](../algorithm_introduction/metric_learning.md)。 + + + +- 图像检索数据集介绍 + + - 训练集合(train dataset)：用来训练模型，使模型能够学习该集合的图像特征。 + - 底库数据集合(gallery dataset)：用来提供图像检索任务中的底库数据，该集合可与训练集或测试集相同，也可以不同，当与训练集相同时，测试集的类别体系应与训练集的类别体系相同。 + - 测试集合(query dataset)：用来测试模型的好坏，通常要对测试集的每一张测试图片进行特征提取，之后和底库数据的特征进行距离匹配，得到识别结果，后根据识别结果计算整个测试集的指标。 + + +- 图像检索评价指标 + + + - 召回率(recall)：表示预测为正例且标签为正例的个数 / 标签为正例的个数 + + - recall@1：检索的 top-1 中预测正例且标签为正例的个数 / 标签为正例的个数 + - recall@5：检索的 top-5 中所有预测正例且标签为正例的个数 / 标签为正例的个数 + + + - 平均检索精度(mAP) + + - AP: AP 指的是不同召回率上的正确率的平均值 + - mAP: 测试集中所有图片对应的 AP 的平均值 diff --git a/src/PaddleClas/docs/zh_CN/models_training/train_strategy.md b/src/PaddleClas/docs/zh_CN/models_training/train_strategy.md new file mode 100644 index 0000000..db5cf1d --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/models_training/train_strategy.md @@ -0,0 +1,137 @@ +# 训练技巧 + +## 目录 + +- [1. 优化器的选择](#1) +- [2. 学习率以及学习率下降策略的选择](#2) + - [2.1 学习率的概念](#2.1) + - [2.2 学习率下降策略](#2.2) + - [2.3 warmup 策略](#2.3) +- [3. batch_size 的选择](#3) +- [4. weight_decay 的选择](#4) +- [5. label_smoothing 的选择](#5) +- [6.针对小模型更改图片的 crop 面积与拉伸变换程度](#6) +- [7.使用数据增广方式提升精度](#7) +- [8. 通过 train_acc 和 test_acc 确定调优策略](#8) +- [9.通过已有的预训练模型提升自己的数据集的精度](#9) + + + +## 1.优化器的选择 +自深度学习发展以来，就有很多关于优化器的研究者工作，优化器的目的是为了让损失函数尽可能的小，从而找到合适的参数来完成某项任务。目前业界主要用到的优化器有 SGD、RMSProp、Adam、AdaDelt 等，其中由于带 momentum 的 SGD 优化器广泛应用于学术界和工业界，所以我们发布的模型也大都使用该优化器来实现损失函数的梯度下降。带 momentum 的 SGD 优化器有两个劣势，其一是收敛速度慢，其二是初始学习率的设置需要依靠大量的经验，然而如果初始学习率设置得当并且迭代轮数充足，该优化器也会在众多的优化器中脱颖而出，使得其在验证集上获得更高的准确率。一些自适应学习率的优化器如 Adam、RMSProp 等，收敛速度往往比较快，但是最终的收敛精度会稍差一些。如果追求更快的收敛速度，我们推荐使用这些自适应学习率的优化器，如果追求更高的收敛精度，我们推荐使用带 momentum 的 SGD 优化器。 + + + +## 2.学习率以及学习率下降策略的选择 +学习率的选择往往和优化器以及数据和任务有关系。这里主要介绍以 momentum+SGD 作为优化器训练 ImageNet-1k 的学习率以及学习率下降的选择。 + + + +### 2.1 学习率的概念： +学习率是通过损失函数的梯度调整网络权重的超参数的速度。学习率越低，损失函数的变化速度就越慢。虽然使用低学习率可以确保不会错过任何局部极小值，但也意味着将花费更长的时间来进行收敛，特别是在被困在高原区域的情况下。 + + + +### 2.2 学习率下降策略： +在整个训练过程中，我们不能使用同样的学习率来更新权重，否则无法到达最优点，所以需要在训练过程中调整学习率的大小。在训练初始阶段，由于权重处于随机初始化的状态，损失函数相对容易进行梯度下降，所以可以设置一个较大的学习率。在训练后期，由于权重参数已经接近最优值，较大的学习率无法进一步寻找最优值，所以需要设置一个较小的学习率。在训练整个过程中，很多研究者使用的学习率下降方式是 piecewise_decay，即阶梯式下降学习率，如在 ResNet50 标准的训练中，我们设置的初始学习率是 0.1，每 30 epoch 学习率下降到原来的 1/10，一共迭代 120 epoch。除了 piecewise_decay，很多研究者也提出了学习率的其他下降方式，如 polynomial_decay（多项式下降）、exponential_decay（指数下降）、cosine_decay（余弦下降）等，其中 cosine_decay 无需调整超参数，鲁棒性也比较高，所以成为现在提高模型精度首选的学习率下降方式。Cosine_decay 和 piecewise_decay 的学习率变化曲线如下图所示，容易观察到，在整个训练过程中，cosine_decay 都保持着较大的学习率，所以其收敛较为缓慢，但是最终的收敛效果较 peicewise_decay 更好一些。 + +![](../../images/models/lr_decay.jpeg) + +另外，从图中我们也可以看到，cosine_decay 里学习率小的轮数较少，这样会影响到最终的精度，所以为了使得 cosine_decay 发挥更好的效果，建议迭代更多的轮数，如 200 轮。 + + + +### 2.3 warmup 策略 +如果使用较大的 batch_size 训练神经网络时，我们建议您使用 warmup 策略。Warmup 策略顾名思义就是让学习率先预热一下，在训练初期我们不直接使用最大的学习率，而是用一个逐渐增大的学习率去训练网络，当学习率增大到最高点时，再使用学习率下降策略中提到的学习率下降方式衰减学习率的值。实验表明，在 batch_size 较大时，warmup 可以稳定提升模型的精度。在训练 MobileNetV3 等 batch_size 较大的实验中，我们默认将 warmup 中的 epoch 设置为 5，即先用 5 epoch 将学习率从 0 增加到最大值，再去做相应的学习率衰减。 + + + +## 3.batch_size 的选择 +batch_size 是训练神经网络中的一个重要的超参数，该值决定了一次将多少数据送入神经网络参与训练。在论文[1]中，作者通过实验发现，当 batch_size 的值与学习率的值呈线性关系时，收敛精度几乎不受影响。在训练 ImageNet 数据时，大部分的神经网络选择的初始学习率为 0.1，batch_size 是 256，所以根据实际的模型大小和显存情况，可以将学习率设置为 0.1\*k, batch_size 设置为 256\*k。 + + + +## 4.weight_decay 的选择 +过拟合是机器学习中常见的一个名词，简单理解即为模型在训练数据上表现很好，但在测试数据上表现较差，在卷积神经网络中，同样存在过拟合的问题，为了避免过拟合，很多正则方式被提出，其中，weight_decay 是其中一个广泛使用的避免过拟合的方式。Weight_decay 等价于在最终的损失函数后添加 L2 正则化，L2 正则化使得网络的权重倾向于选择更小的值，最终整个网络中的参数值更趋向于 0，模型的泛化性能相应提高。在各大深度学习框架的实现中，该值表达的含义是 L2 正则前的系数，在 paddle 框架中，该值的名称是 l2_decay，所以以下都称其为 l2_decay。该系数越大，表示加入的正则越强，模型越趋于欠拟合状态。在训练 ImageNet 的任务中，大多数的网络将该参数值设置为 1e-4，在一些小的网络如 MobileNet 系列网络中，为了避免网络欠拟合，该值设置为 1e-5~4e-5 之间。当然，该值的设置也和具体的数据集有关系，当任务的数据集较大时，网络本身趋向于欠拟合状态，可以将该值适当减小，当任务的数据集较小时，网络本身趋向于过拟合状态，可以将该值适当增大。下表展示了 MobileNetV1_x0_25 在 ImageNet-1k 上使用不同 l2_decay 的精度情况。由于 MobileNetV1_x0_25 是一个比较小的网络，所以 l2_decay 过大会使网络趋向于欠拟合状态，所以在该网络中，相对 1e-4, 3e-5 是更好的选择。 + +| 模型 | L2_decay | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV1_x0_25 | 1e-4 | 43.79%/67.61% | 50.41%/74.70% | +| MobileNetV1_x0_25 | 3e-5 | 47.38%/70.83% | 51.45%/75.45% | + +另外，该值的设置也和训练过程中是否使用其他正则化有关系。如果训练过程中的数据预处理比较复杂，相当于训练任务变的更难，可以将该值适当减小，下表展示了在 ImageNet-1k 上, ResNet50 在使用 randaugment 预处理方式后使用不同 l2_decay 的精度。容易观察到，在任务变难后，使用更小的 l2_decay 有助于模型精度的提升。 + +| 模型 | L2_decay | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| ResNet50 | 1e-4 | 75.13%/90.42% | 77.65%/93.79% | +| ResNet50 | 7e-5 | 75.56%/90.55% | 78.04%/93.74% | + +综上所述，l2_decay 可以根据具体的任务和模型去做相应的调整，通常简单的任务或者较大的模型，推荐使用较大的 l2_decay，复杂的任务或者较小的模型，推荐使用较小的 l2_decay。 + + + +## 5.label_smoothing 的选择 +Label_smoothing 是深度学习中的一种正则化方法，其全称是 Label Smoothing Regularization(LSR)，即标签平滑正则化。在传统的分类任务计算损失函数时，是将真实的 one hot 标签与神经网络的输出做相应的交叉熵计算，而 label_smoothing 是将真实的 one hot 标签做一个标签平滑的处理，使得网络学习的标签不再是一个 hard label，而是一个有概率值的 soft label，其中在类别对应的位置的概率最大，其他位置概率是一个非常小的数。具体的计算方式参见论文[2]。在 label_smoothing 里，有一个 epsilon 的参数值，该值描述了将标签软化的程度，该值越大，经过 label smoothing 后的标签向量的标签概率值越小，标签越平滑，反之，标签越趋向于 hard label，在训练 ImageNet-1k 的实验里通常将该值设置为 0.1。 +在训练 ImageNet-1k 的实验中，我们发现，ResNet50 大小级别及其以上的模型在使用 label_smooting 后，精度有稳定的提升。下表展示了 ResNet50_vd 在使用 label_smoothing 前后的精度指标。 + +| 模型 | Use_label_smoothing | Test acc1 | +|:--:|:--:|:--:| +| ResNet50_vd | 0 | 77.9% | +| ResNet50_vd | 1 | 78.4% | + +同时，由于 label_smoohing 相当于一种正则方式，在相对较小的模型上，精度提升不明显甚至会有所下降，下表展示了 ResNet18 在 ImageNet-1k 上使用 label_smoothing 前后的精度指标。可以明显看到，在使用 label_smoothing 后，精度有所下降。 + +| 模型 | Use_label_smoohing | Train acc1/acc5 | Test acc1/acc5 | +|:--:|:--:|:--:|:--:| +| ResNet18 | 0 | 69.81%/87.70% | 70.98%/89.92% | +| ResNet18 | 1 | 68.00%/86.56% | 70.81%/89.89% | + +综上所述，较大的模型使用 label_smoohing 可以有效提升模型的精度，较小的模型使用 label_smoohing 可能会降低模型的精度，所以在决定是否使用 label_smoohing 前，需要评估模型的大小和任务的难易程度。 + + + +## 6.针对小模型更改图片的 crop 面积与拉伸变换程度 +在 ImageNet-1k 数据的标准预处理中，random_crop 函数中定义了 scale 和 ratio 两个值，两个值分别确定了图片 crop 的大小和图片的拉伸程度，其中 scale 的默认取值范围是 0.08-1(lower_scale-upper_scale), ratio 的默认取值范围是 3/4-4/3(lower_ratio-upper_ratio)。在非常小的网络训练中，此类数据增强会使得网络欠拟合，导致精度有所下降。为了提升网络的精度，可以使其数据增强变的更弱，即增大图片的 crop 区域或者减弱图片的拉伸变换程度。我们可以分别通过增大 lower_scale 的值或缩小 lower_ratio 与 upper_scale 的差距来实现更弱的图片变换。下表列出了使用不同 lower_scale 训练 MobileNetV2_x0_25 的精度，可以看到，增大图片的 crop 区域面积后训练精度和验证精度均有提升。 + +| 模型 | Scale 取值范围 | Train_acc1/acc5 | Test_acc1/acc5 | +|:--:|:--:|:--:|:--:| +| MobileNetV2_x0_25 | [0.08,1] | 50.36%/72.98% | 52.35%/75.65% | +| MobileNetV2_x0_25 | [0.2,1] | 54.39%/77.08% | 53.18%/76.14% | + + + +## 7. 使用数据增广方式提升精度 + +一般来说，数据集的规模对性能影响至关重要，但是图片的标注往往比较昂贵，所以有标注的图片数量往往比较稀少，在这种情况下，数据的增广尤为重要。在训练 ImageNet-1k 的标准数据增广中，主要使用了 random_crop 与 random_flip 两种数据增广方式，然而，近些年，越来越多的数据增广方式被提出，如 cutout、mixup、cutmix、AutoAugment 等。实验表明，这些数据的增广方式可以有效提升模型的精度，下表列出了 ResNet50 在 8 种不同的数据增广方式的表现，可以看出，相比 baseline，所有的数据增广方式均有收益，其中 cutmix 是目前最有效的数据增广。更多数据增广的介绍请参考[**数据增广章节**](../algorithm_introduction/DataAugmentation.md)。 + +| 模型 | 数据增广方式 | Test top-1 | +|:--:|:--:|:--:| +| ResNet50 | 标准变换 | 77.31% | +| ResNet50 | Auto-Augment | 77.95% | +| ResNet50 | Mixup | 78.28% | +| ResNet50 | Cutmix | 78.39% | +| ResNet50 | Cutout | 78.01% | +| ResNet50 | Gridmask | 77.85% | +| ResNet50 | Random-Augment | 77.70% | +| ResNet50 | Random-Erasing | 77.91% | +| ResNet50 | Hide-and-Seek | 77.43% | + + + +## 8. 通过 train_acc 和 test_acc 确定调优策略 + +在训练网络的过程中，通常会打印每一个 epoch 的训练集准确率和验证集准确率，二者刻画了该模型在两个数据集上的表现。通常来说，训练集的准确率比验证集准确率微高或者二者相当是比较不错的状态。如果发现训练集的准确率比验证集高很多，说明在这个任务上已经过拟合，需要在训练过程中加入更多的正则，如增大 l2_decay 的值，加入更多的数据增广策略，加入 label_smoothing 策略等；如果发现训练集的准确率比验证集低一些，说明在这个任务上可能欠拟合，需要在训练过程中减弱正则效果，如减小 l2_decay 的值，减少数据增广方式，增大图片 crop 区域面积，减弱图片拉伸变换，去除 label_smoothing 等。 + + + +## 9.通过已有的预训练模型提升自己的数据集的精度 + +在现阶段计算机视觉领域中，加载预训练模型来训练自己的任务已成为普遍的做法，相比从随机初始化开始训练，加载预训练模型往往可以提升特定任务的精度。一般来说，业界广泛使用的预训练模型是通过训练 128 万张图片 1000 类的 ImageNet-1k 数据集得到的，该预训练模型的 fc 层权重是一个 k\*1000 的矩阵，其中 k 是 fc 层以前的神经元数，在加载预训练权重时，无需加载 fc 层的权重。在学习率方面，如果您的任务训练的数据集特别小（如小于 1 千张），我们建议你使用较小的初始学习率，如 0.001（batch_size:256,下同），以免较大的学习率破坏预训练权重。如果您的训练数据集规模相对较大（大于 10 万），我们建议你尝试更大的初始学习率，如 0.01 或者更大。 + + +> 如果您觉得此文档对您有帮助，欢迎 star 我们的项目：[https://github.com/PaddlePaddle/PaddleClas](https://github.com/PaddlePaddle/PaddleClas) + +## 参考文献 +[1]P. Goyal, P. Dolla ́r, R. B. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. Accurate, large minibatch SGD: training imagenet in 1 hour. CoRR, abs/1706.02677, 2017. + +[2]C.Szegedy,V.Vanhoucke,S.Ioffe,J.Shlens,andZ.Wojna. Rethinking the inception architecture for computer vision. CoRR, abs/1512.00567, 2015. diff --git a/src/PaddleClas/docs/zh_CN/others/VisualDL.md b/src/PaddleClas/docs/zh_CN/others/VisualDL.md new file mode 100644 index 0000000..b1ccb96 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/VisualDL.md @@ -0,0 +1,56 @@ +# 使用 VisualDL 可视化训练过程 + +--- + +## 目录 +* [1. 前言](#1) +* [2. 在 PaddleClas 中使用 VisualDL](#2) + * [2.1 设置 config 文件并启动训练](#2.1) + * [2.2 启动 VisualDL](#2.2) + + + +## 1. 前言 +VisualDL 是飞桨可视化分析工具，以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构，进而实现高效的模型优化。更多细节请查看[VisualDL](https://github.com/PaddlePaddle/VisualDL/)。 + + + +## 2. 在 PaddleClas 中使用 VisualDL +现在 PaddleClas 支持在训练阶段使用 VisualDL 查看训练过程中学习率（learning rate）、损失值（loss）以及准确率（accuracy）的变化情况。 + + + +### 2.1 设置 config 文件并启动训练 +在 PaddleClas 中使用 VisualDL，只需在训练配置文件（config 文件）中设置字段 `Global.use_visualdl` 为 `True`： + +```yaml +# config.yaml +Global: +... + use_visualdl: True +... +``` + +PaddleClas 会将 VisualDL 的日志保存在 `Global.output_dir` 字段指定目录下的 `vdl/` 子目录下，然后正常启动训练即可： + +```shell +python3 tools/train.py -c config.yaml +``` + + +### 2.2 启动 VisualDL +在启动训练程序后，可以在新的终端 session 中启动 VisualDL 服务： + +```shell + visualdl --logdir ./output/vdl/ + ``` + +上述命令中，参数`--logdir` 用于指定保存 VisualDL 日志的目录，VisualDL 将遍历并且迭代寻找指定目录的子目录，将所有实验结果进行可视化。也同样可以使用下述参数设定 VisualDL 服务的 ip 及端口号： +* `--host`：设定 IP，默认为 127.0.0.1 +* `--port`：设定端口，默认为 8040 + +更多参数信息，请查看[VisualDL](https://github.com/PaddlePaddle/VisualDL/blob/develop/README_CN.md#2-%E5%90%AF%E5%8A%A8%E9%9D%A2%E6%9D%BF)。 + +在启动 VisualDL 后，即可在浏览器中查看训练过程，输入地址 `127.0.0.1:8840`： + +![](../../images/VisualDL/train_loss.png) diff --git a/src/PaddleClas/docs/zh_CN/others/competition_support.md b/src/PaddleClas/docs/zh_CN/others/competition_support.md new file mode 100644 index 0000000..f20eba4 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/competition_support.md @@ -0,0 +1,19 @@ +### 赛事支持 + +PaddleClas 的建设源于百度实际视觉业务应用的淬炼和视觉前沿能力的探索，助力多个视觉重点赛事取得领先成绩，并且持续推进更多的前沿视觉问题的解决和落地应用。 + +* 2018 年 Kaggle Open Images V4 图像目标检测挑战赛冠军 + +* 2019 年 Kaggle Open Images V5 图像目标检测挑战赛亚军 + * 技术报告可以参考：[https://arxiv.org/pdf/1911.07171.pdf](https://arxiv.org/pdf/1911.07171.pdf) + * 详细文档与开源的模型可以参考：[OIDV5目标检测 github 地址](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.2/static/docs/featured_model/champion_model/OIDV5_BASELINE_MODEL.md) + +* 2019 年 Kaggle 地标检索挑战赛亚军 + * 技术报告可以参考：[https://arxiv.org/abs/1906.03990](https://arxiv.org/abs/1906.03990) + * 详细文档与开源的模型可以参考：[2019地标检索和识别 github 地址](https://github.com/PaddlePaddle/Research/tree/master/CV/landmark) + +* 2019 年 Kaggle 地标识别挑战赛亚军 + * 技术报告可以参考：[https://arxiv.org/abs/1906.03990](https://arxiv.org/abs/1906.03990) + * 详细文档与开源的模型可以参考：[2019地标检索和识别 github 地址](https://github.com/PaddlePaddle/Research/tree/master/CV/landmark) + +* 首届多媒体信息识别技术竞赛中印刷文本 OCR、人脸识别和地标识别三项任务 A 级证书 diff --git a/src/PaddleClas/docs/zh_CN/others/course_link.md b/src/PaddleClas/docs/zh_CN/others/course_link.md new file mode 100644 index 0000000..1776320 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/course_link.md @@ -0,0 +1,34 @@ +# 往期课程链接 +--- + +- [**【AI快车道PaddleClas系列直播课】**](https://aistudio.baidu.com/aistudio/course/introduce/24519) + + - [图像识别系统解析](https://aistudio.baidu.com/aistudio/education/group/info/24519) + - 图像识别全能优势 + - 整体架构及快速落地详解 + - 个性化需求实现方案 + - [商品识别系统全拆解](https://aistudio.baidu.com/aistudio/education/lessonvideo/1495317) + - 小样本多类别场景方案 + - 图像检索技术及快速构建策略 + - 动漫搜索趣味应用 + - [车辆ReID核心技术方案](https://aistudio.baidu.com/aistudio/education/lessonvideo/1496537) + - ReID及跨境头场景应用 + - Metric Learning——更鲁棒的检索特征 + - Logo识别等方向延展 + - [超轻量图像识别系统概览](https://aistudio.baidu.com/aistudio/education/lessonvideo/1890318) + - 图像识别技术选型策略 + - 推理速度提升8倍的秘籍 + - 四大典型行业应用案例 + - [SOTA模型炼丹秘诀](https://aistudio.baidu.com/aistudio/education/lessonvideo/1890323) + - CPU定制模型PP-LCNet优化思路 + - Vison Transformer模型的应用拓展 + - [商品识别产业痛点剖析](https://aistudio.baidu.com/aistudio/education/lessonvideo/1896890) + - 特征提取技术详解 + - 向量快速检索揭秘 + - [手把手教你玩转图像识别](https://aistudio.baidu.com/aistudio/education/lessonvideo/1911507) + - 产业应用十问十答 + - 智能零售下的应用案例 + - 识别系统快速落地方案 + + + diff --git a/src/PaddleClas/docs/zh_CN/others/feature_visiualization.md b/src/PaddleClas/docs/zh_CN/others/feature_visiualization.md new file mode 100644 index 0000000..4406e1a --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/feature_visiualization.md @@ -0,0 +1,102 @@ +# 特征图可视化指南 +----- +## 目录 + +* [1. 概述](#1) +* [2. 准备工作](#2) +* [3. 修改模型](#3) +* [4. 结果](#4) + + + +## 1. 概述 + +特征图是输入图片在卷积网络中的特征表达，对特征图的研究可以有利于我们对于模型的理解与设计，所以基于动态图我们使用本工具来可视化特征图。 + + + +## 2. 准备工作 + +首先需要选定研究的模型，本文设定 ResNet50 作为研究模型，将模型组网代码[resnet.py](../../../ppcls/arch/backbone/legendary_models/resnet.py)拷贝到[目录](../../../ppcls/utils/feature_maps_visualization/)下，并下载[ResNet50 预训练模型](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNet50_pretrained.pdparams)，或使用以下命令下载。 + +```bash +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ResNet50_pretrained.pdparams +``` + +其他模型网络结构代码及预训练模型请自行下载：[模型库](../../../ppcls/arch/backbone/)，[预训练模型](../models/models_intro.md)。 + + + +## 3. 修改模型 + +找到我们所需要的特征图位置，设置 self.fm 将其 fetch 出来，本文以 resnet50 中的 stem 层之后的特征图为例。 + +在 ResNet50 的 forward 函数中指定要可视化的特征图 + +```python + def forward(self, x): + with paddle.static.amp.fp16_guard(): + if self.data_format == "NHWC": + x = paddle.transpose(x, [0, 2, 3, 1]) + x.stop_gradient = True + x = self.stem(x) + fm = x + x = self.max_pool(x) + x = self.blocks(x) + x = self.avg_pool(x) + x = self.flatten(x) + x = self.fc(x) + return x, fm +``` + +然后修改代码[fm_vis.py](../../../ppcls/utils/feature_maps_visualization/fm_vis.py)，引入 `ResNet50`，实例化 `net` 对象： + +```python +from resnet import ResNet50 +net = ResNet50() +``` + +最后执行函数 + +```bash +python tools/feature_maps_visualization/fm_vis.py \ + -i the image you want to test \ + -c channel_num -p pretrained model \ + --show whether to show \ + --interpolation interpolation method\ + --save_path where to save \ + --use_gpu whether to use gpu +``` + +参数说明： ++ `-i`：待预测的图片文件路径，如 `./test.jpeg` ++ `-c`：特征图维度，如 `5` ++ `-p`：权重文件路径，如 `./ResNet50_pretrained` ++ `--interpolation`: 图像插值方式，默认值 1 ++ `--save_path`：保存路径，如：`./tools/` ++ `--use_gpu`：是否使用 GPU 预测，默认值：True + + + +## 4. 结果 + +* 输入图片： + +![](../../images/feature_maps/feature_visualization_input.jpg) + +* 运行下面的特征图可视化脚本 + +``` +python tools/feature_maps_visualization/fm_vis.py \ + -i ./docs/images/feature_maps/feature_visualization_input.jpg \ + -c 5 \ + -p pretrained/ResNet50_pretrained/ \ + --show=True \ + --interpolation=1 \ + --save_path="./output.png" \ + --use_gpu=False +``` + +* 输出特征图保存为 `output.png`，如下所示。 + +![](../../images/feature_maps/feature_visualization_output.jpg) diff --git a/src/PaddleClas/docs/zh_CN/others/index.rst b/src/PaddleClas/docs/zh_CN/others/index.rst new file mode 100644 index 0000000..1141b41 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/index.rst @@ -0,0 +1,20 @@ +其他 +================================ + +.. toctree:: + :maxdepth: 2 + + transfer_learning.md + train_with_DALI.md + VisualDL.md + train_on_xpu.md + feature_visiualization.md + paddle_mobile_inference.md + course_link.md + competition_support.md + update_history.md + versions.md + + + + diff --git a/src/PaddleClas/docs/zh_CN/others/paddle_mobile_inference.md b/src/PaddleClas/docs/zh_CN/others/paddle_mobile_inference.md new file mode 100644 index 0000000..6cdb19a --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/paddle_mobile_inference.md @@ -0,0 +1,137 @@ +# 手机端 benchmark +--- +## 目录 + +* [1. 简介](#1) +* [2. 评估步骤](#2) + * [2.1 导出 inference 模型](#2.1) + * [2.2 benchmark 二进制文件下载](#2.2) + * [2.3 模型速度 benchmark](#2.3) + * [2.4 模型优化与速度评估](#2.4) + + + +## 1. 简介 + +[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) 是飞桨推出的一套功能完善、易用性强且性能卓越的轻量化推理引擎。 +轻量化体现在使用较少比特数用于表示神经网络的权重和激活，能够大大降低模型的体积，解决终端设备存储空间有限的问题，推理性能也整体优于其他框架。 +[PaddleClas](https://github.com/PaddlePaddle/PaddleClas) 使用 Paddle-Lite 进行了[移动端模型的性能评估](../models/Mobile.md)，本部分以 `ImageNet1k` 数据集的 `MobileNetV1` 模型为例，介绍怎样使用 `Paddle-Lite`，在移动端(基于骁龙855的安卓开发平台)对进行模型速度评估。 + + + +## 2. 评估步骤 + + + +### 2.1 导出 inference 模型 + +* 首先需要将训练过程中保存的模型存储为用于预测部署的固化模型，可以使用 `tools/export_model.py` 导出 inference 模型，具体使用方法如下。 + +```shell +python tools/export_model.py \ + -c ./ppcls/configs/ImageNet/MobileNetV1/MobileNetV1.yaml \ + -o Arch.pretrained=./pretrained/MobileNetV1_pretrained/ \ + -o Global.save_inference_dir=./inference/MobileNetV1/ +``` + +在上述命令中，通过参数 `Arch.pretrained` 指定训练过程中保存的模型参数文件，也可以指定参数 `Arch.pretrained=True` 加载 PaddleClas 提供的基于 ImageNet1k 的预训练模型参数，最终在 `inference/MobileNetV1` 文件夹下会保存得到 `inference.pdmodel` 与 `inference.pdiparmas` 文件。 + + + +### 2.2 benchmark 二进制文件下载 + +* 使用 adb(Android Debug Bridge)工具可以连接 Android 手机与 PC 端，并进行开发调试等。安装好 adb，并确保 PC 端和手机连接成功后，使用以下命令可以查看手机的 ARM 版本，并基于此选择合适的预编译库。 + +```shell +adb shell getprop ro.product.cpu.abi +``` + +* 下载 benchmark_bin 文件 + +请根据所用 Android 手机的 ARM 版本选择，ARM 版本为 v8，则使用以下命令下载： + +```shell +wget -c https://paddle-inference-dist.bj.bcebos.com/PaddleLite/benchmark_0/benchmark_bin_v8 +``` + +如果查看的 ARM 版本为 v7，则需要下载 v7 版本的 benchmark_bin 文件，下载命令如下： + +```shell +wget -c https://paddle-inference-dist.bj.bcebos.com/PaddleLite/benchmark_0/benchmark_bin_v7 +``` + + + +### 2.3 模型速度 benchmark + +PC 端和手机连接成功后，使用下面的命令开始模型评估。 + +``` +sh deploy/lite/benchmark/benchmark.sh ./benchmark_bin_v8 ./inference result_armv8.txt true +``` + +其中 `./benchmark_bin_v8` 为 benchmark 二进制文件路径，`./inference` 为所有需要评测的模型的路径，`result_armv8.txt` 为保存的结果文件，最后的参数 `true` 表示在评估之后会首先进行模型优化。最终在当前文件夹下会输出 `result_armv8.txt` 的评估结果文件，具体信息如下。 + +``` +PaddleLite Benchmark +Threads=1 Warmup=10 Repeats=30 +MobileNetV1 min = 30.89100 max = 30.73600 average = 30.79750 + +Threads=2 Warmup=10 Repeats=30 +MobileNetV1 min = 18.26600 max = 18.14000 average = 18.21637 + +Threads=4 Warmup=10 Repeats=30 +MobileNetV1 min = 10.03200 max = 9.94300 average = 9.97627 +``` + +这里给出了不同线程数下的模型预测速度，单位为 FPS，以线程数为 1 为例，MobileNetV1 在骁龙855上的平均速度为 `30.79750FPS`。 + + + +### 2.4 模型优化与速度评估 + + +* 在 2.3 节中提到了在模型评估之前对其进行优化，在这里也可以首先对模型进行优化，再直接加载优化后的模型进行速度评估。 + +* Paddle-Lite 提供了多种策略来自动优化原始的训练模型，其中包括量化、子图融合、混合调度、Kernel 优选等等方法。为了使优化过程更加方便易用，Paddle-Lite 提供了 opt 工具来自动完成优化步骤，输出一个轻量的、最优的可执行模型。可以在[Paddle-Lite 模型优化工具页面](https://paddle-lite.readthedocs.io/zh/latest/user_guides/model_optimize_tool.html)下载。在这里以 `macOS` 开发环境为例，下载[opt_mac](https://paddlelite-data.bj.bcebos.com/model_optimize_tool/opt_mac)模型优化工具，并使用下面的命令对模型进行优化。 + + + +```shell +model_file="../MobileNetV1/inference.pdmodel" +param_file="../MobileNetV1/inference.pdiparams" +opt_models_dir="./opt_models" +mkdir ${opt_models_dir} +./opt_mac --model_file=${model_file} \ + --param_file=${param_file} \ + --valid_targets=arm \ + --optimize_out_type=naive_buffer \ + --prefer_int8_kernel=false \ + --optimize_out=${opt_models_dir}/MobileNetV1 +``` + +其中 `model_file` 与 `param_file` 分别是导出的 inference 模型结构文件与参数文件地址，转换成功后，会在 `opt_models` 文件夹下生成 `MobileNetV1.nb` 文件。 + +使用 benchmark_bin 文件加载优化后的模型进行评估，具体的命令如下。 + +```shell +bash benchmark.sh ./benchmark_bin_v8 ./opt_models result_armv8.txt +``` + +最终 `result_armv8.txt` 中结果如下： + +``` +PaddleLite Benchmark +Threads=1 Warmup=10 Repeats=30 +MobileNetV1_lite min = 30.89500 max = 30.78500 average = 30.84173 + +Threads=2 Warmup=10 Repeats=30 +MobileNetV1_lite min = 18.25300 max = 18.11000 average = 18.18017 + +Threads=4 Warmup=10 Repeats=30 +MobileNetV1_lite min = 10.00600 max = 9.90000 average = 9.96177 +``` + +以线程数为 1 为例，MobileNetV1 在骁龙855上的平均速度为 `30.84173 ms`。 + +更加具体的参数解释与 Paddle-Lite 使用方法可以参考 [Paddle-Lite 文档](https://paddle-lite.readthedocs.io/zh/latest/)。 diff --git a/src/PaddleClas/docs/zh_CN/others/train_on_xpu.md b/src/PaddleClas/docs/zh_CN/others/train_on_xpu.md new file mode 100644 index 0000000..429119b --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/train_on_xpu.md @@ -0,0 +1,81 @@ +# 图像分类昆仑模型介绍(持续更新中) +------ +## 目录 +* [1. 前言](#1) +* [2. 昆仑训练](#2) + * [2.1 ResNet50](#2.1) + * [2.2 MobileNetV3](#2.2) + * [2.3 HRNet](#2.3) + * [2.4 VGG16/19](#2.4) + + + +## 1. 前言 + +* 本文档介绍了目前昆仑支持的模型以及如何在昆仑设备上训练这些模型。支持昆仑的 PaddlePaddle 安装参考 install_kunlun(https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/paddle/install/install_Kunlun_zh.md) + + + +## 2. 昆仑训练 +* 数据来源和预训练模型参考[quick_start](../quick_start/quick_start_classification_new_user.md)。昆仑训练效果与 CPU/GPU 对齐。 + + + +### 2.1 ResNet50 +* 命令： + +```shell +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/ResNet50_vd_finetune_kunlun.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +``` + +与 cpu/gpu 训练的区别是加上 -o use_xpu=True, 表示执行在昆仑设备上。 + + + +### 2.2 MobileNetV3 +* 命令： + +```shell +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/MobileNetV3_large_x1_0.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +``` + + + +### 2.3 HRNet +* 命令： + +```shell +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/HRNet_W18_C_finetune_kunlun.yaml \ + -o is_distributed=False \ + -o use_xpu=True \ + -o use_gpu=False +``` + + + +### 2.4 VGG16/19 +* 命令： + +```shell +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/VGG16_finetune_kunlun.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +``` +```shell +python3.7 ppcls/static/train.py \ + -c ppcls/configs/quick_start/kunlun/VGG19_finetune_kunlun.yaml \ + -o use_gpu=False \ + -o use_xpu=True \ + -o is_distributed=False +``` diff --git a/src/PaddleClas/docs/zh_CN/others/train_with_DALI.md b/src/PaddleClas/docs/zh_CN/others/train_with_DALI.md new file mode 100644 index 0000000..34782ea --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/train_with_DALI.md @@ -0,0 +1,72 @@ +# 使用 DALI 加速训练 +---- +## 目录 +* [1. 前言](#1) +* [2. 安装 DALI](#2) +* [3. 使用 DALI](#3) +* [4. 使用 FP16 训练](#4) + + + +## 1. 前言 +[NVIDIA 数据加载库](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html)（The NVIDIA Data Loading Library，DALI）是用于数据加载和预处理的开源库，用于加速深度学习训练、推理过程，它可以直接构建飞桨 Paddle 的 DataLoader 数据读取器。 + +由于深度学习程序在训练阶段依赖大量数据，这些数据需要经过加载、预处理等操作后，才能送入训练程序，而这些操作通常在 CPU 完成，因此限制了训练速度进一步提高，特别是在 batch_size 较大时，数据读取可能成为训练速度的瓶颈。 DALI 可以基于 GPU 的高并行特性实现数据加载及预处理操作，可以进一步提高训练速度。 + + + +## 2.安装 DALI +目前 DALI 仅支持 Linux x64 平台，且 CUDA 版本大于等于 10.2。 + +* 对于 CUDA 10: + + pip install --extra-index-url https://developer.download.nvidia.com/compute/redist nvidia-dali-cuda100 + +* 对于 CUDA 11.0: + + pip install --extra-index-url https://developer.download.nvidia.com/compute/redist nvidia-dali-cuda110 + +关于更多 DALI 安装的信息，可以参考[DALI 官方](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html)。 + + + +## 3. 使用 DALI +PaddleClas 支持使用 DALI 对图像预处理进行加速，由于 DALI 仅支持 GPU 训练，因此需要设置 GPU，且 DALI 需要占用 GPU 显存，需要为 DALI 预留显存。使用 DALI 训练只需在训练配置文件中设置字段 `use_dali=True`，或通过以下命令启动训练即可： + +```shell +# 设置用于训练的 GPU 卡号 +export CUDA_VISIBLE_DEVICES="0" + +python ppcls/train.py -c ppcls/configs/ImageNet/ResNet/ResNet50.yaml -o Global.use_dali=True +``` + +也可以使用多卡训练： + +```shell +# 设置用于训练的 GPU 卡号 +export CUDA_VISIBLE_DEVICES="0,1,2,3" + +# 设置用于神经网络训练的显存大小，可根据具体情况设置，一般可设置为 0.8 或 0.7，剩余显存则预留 DALI 使用 +export FLAGS_fraction_of_gpu_memory_to_use=0.80 + +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + ppcls/train.py \ + -c ./ppcls/configs/ImageNet/ResNet/ResNet50.yaml \ + -o Global.use_dali=True +``` + + + +## 4. 使用 FP16 训练 +在上述基础上，使用 FP16 半精度训练，可以进一步提高速度，可以参考下面的配置与运行命令。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +export FLAGS_fraction_of_gpu_memory_to_use=0.8 + +python -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + ppcls/train.py \ + -c ./ppcls/configs/ImageNet/ResNet/ResNet50_fp16_dygraph.yaml +``` diff --git a/src/PaddleClas/docs/zh_CN/others/transfer_learning.md b/src/PaddleClas/docs/zh_CN/others/transfer_learning.md new file mode 100644 index 0000000..d42bc09 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/transfer_learning.md @@ -0,0 +1,103 @@ +# 图像分类迁移学习 +迁移学习是机器学习领域的一个重要分支，广泛应用于文本、图像等各种领域，此处我们主要介绍的是图像分类领域的迁移学习，也就是我们常说的域迁移，比如将 ImageNet 分类模型迁移到我们自己场景的图像分类任务上，如花卉分类。 + +---------- + +## 目录 + +* [1. 超参搜索](#1) + * [1.1 网格搜索](#1.1) + * [1.2 贝叶斯搜索](#1.2) +* [2. 大规模分类模型](#2) +* [3. 参考文献](#3) + + + +## 1. 超参搜索 + +ImageNet 作为业界常用的图像分类数据被大家广泛使用，已经总结出一系列经验性的超参，使用这些超参往往能够得到不错的训练精度，而这些经验性的参数在迁移到自己的业务中时，有时效果不佳。有两种常用的超参搜索方法可以用于获得更好的模型超参。 + + + +### 1.1 网格搜索 + +网格搜索，即穷举搜索，通过查找搜索空间内所有的点，确定最优值。方法简单有效，但当搜索空间较大时，需要消耗大量的计算资源。 + + + +### 1.2 贝叶斯搜索 + +贝叶斯搜索，即贝叶斯优化，在搜索空间中随机选取超参数点，采用高斯过程，即根据上一个超参数点的结果，更新当前的先验信息，计算前面 n 个超参数点的后验概率分布，得到搜索空间中每一个超参数点的期望均值和方差，其中期望均值越大表示接近最优指标的可能性越大，方差越大表示不确定性越大。通常将选择期望均值大的超参数点称为 `exporitation`，选择方差大的超参数点称为 `exploration`。在贝叶斯优化中通过定义 `acquisition function` 权衡期望均值和方差。贝叶斯搜索认为当前选择的超参数点是处于最大值可能出现的位置。 + +------ + +基于上述两种搜索方案，我们在 8 个开源数据集上将固定一组参数实验以及两种搜索方案做了对比实验，参照[1]的实验方案，我们对 4 个超参数进行搜索，搜索空间及实验结果如下所示： + +- 固定参数： + +``` +初始学习率 lr=0.003，l2 decay=1e-4，label smoothing=False，mixup=False +``` + +- 超参搜索空间： + +``` +初始学习率 lr: [0.1, 0.03, 0.01, 0.003, 0.001, 0.0003, 0.0001] + +L2 decay: [1e-3, 3e-4, 1e-4, 3e-5, 1e-5, 3e-6, 1e-6] + +Label smoothing: [False, True] + +Mixup: [False, True] +``` + +网格搜索的搜索次数为 196 次，而贝叶斯搜索通过设置最大迭代次数（`max_iter`）和是否重复搜索（`de_duplication`）来确定搜索次数。我们设计了系列实验，baseline 为 ImageNet1k 校验集 Top1 Acc 为 79.12% 的 ResNet50_vd 预训练模型，并固定超参，在新数据集上 finetune 得到的模型。下表给出了固定参数、网格搜索以及贝叶斯搜索的精度与搜索次数对比。 + +- 精度与搜索次数对比: + +| 数据集 | 固定参数 | 网格搜索 | 网格搜索次数 | 贝叶斯搜索 | 贝叶斯搜索次数| +| ------------------ | -------- | -------- | -------- | -------- | ---------- | +| Oxford-IIIT-Pets | 93.64% | 94.55% | 196 | 94.04% | 20 | +| Oxford-102-Flowers | 96.08% | 97.69% | 196 | 97.49% | 20 | +| Food101 | 87.07% | 87.52% | 196 | 87.33% | 23 | +| SUN397 | 63.27% | 64.84% | 196 | 64.55% | 20 | +| Caltech101 | 91.71% | 92.54% | 196 | 92.16% | 14 | +| DTD | 76.87% | 77.53% | 196 | 77.47% | 13 | +| Stanford Cars | 85.14% | 92.72% | 196 | 92.72% | 25 | +| FGVC Aircraft | 80.32% | 88.45% | 196 | 88.36% | 20 | + + +- 上述实验验证了贝叶斯搜索相比网格搜索，在减少搜索次数 10 倍左右条件下，精度只下降 0%~0.4%。 +- 当搜索空间进一步扩大时，例如将是否进行 AutoAugment，RandAugment，Cutout，Cutmix 以及 Dropout 这些正则化策略作为选择时，贝叶斯搜索能够在获取较优精度的前提下，有效地降低搜索次数。 + + + +## 2. 大规模分类模型 + +在实际应用中，由于训练数据的匮乏，往往将 ImageNet1k 数据集训练的分类模型作为预训练模型，进行图像分类的迁移学习。为了进一步助力解决实际问题，基于 ResNet50_vd, 百度开源了自研的大规模分类预训练模型，其中训练数据为 10 万个类别，4300 万张图片。10 万类预训练模型的下载地址：[**下载地址**](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/legendary_models/ResNet50_vd_10w_pretrained.pdparams) + +我们在 6 个自有采集的数据集上进行迁移学习实验，采用一组固定参数以及网格搜索方式，其中训练轮数设置为 20 epochs，选用 ResNet50_vd 模型，ImageNet 预训练精度为 79.12%。实验数据集参数以及模型精度的对比结果如下： + +固定参数： + +``` +初始学习率 lr=0.001，l2 decay=1e-4，label smoothing=False，mixup=False +``` + +| 数据集 | 数据统计 | **ImageNet 预训练模型
固定参数 Top-1/参数搜索 Top-1** | **大规模分类预训练模型
固定参数 Top-1/参数搜索 Top-1** | +| --------------- | ----------------------------------------- | -------------------------------------------------------- | --------------------------------------------------------- | +| 花卉 | class:102
train:5789
valid:2396 | 0.7779/0.9883 | 0.9892/0.9954 | +| 手绘简笔画 | Class:18
train:1007
valid:432 | 0.8795/0.9196 | 0.9107/0.9219 | +| 植物叶子 | class:6
train:5256
valid:2278 | 0.8212/0.8482 | 0.8385/0.8659 | +| 集装箱车辆 | Class:115
train:4879
valid:2094 | 0.6230/0.9556 | 0.9524/0.9702 | +| 椅子 | class:5
train:169
valid:78 | 0.8557/0.9688 | 0.9077/0.9792 | +| 地质 | class:4
train:671
valid:296 | 0.5719/0.8094 | 0.6781/0.8219 | + +- 通过上述的实验验证了当使用一组固定参数时，相比于 ImageNet 预训练模型，使用大规模分类模型作为预训练模型在大多数情况下能够提升模型在新的数据集上得效果，通过参数搜索可以进一步提升精度。 + + +## 3. 参考文献 + +[1] Kornblith, Simon, Jonathon Shlens, and Quoc V. Le. "Do better imagenet models transfer better?." *Proceedings of the IEEE conference on computer vision and pattern recognition*. 2019. + +[2] Kolesnikov, Alexander, et al. "Large Scale Learning of General Visual Representations for Transfer." *arXiv preprint arXiv:1912.11370* (2019). diff --git a/src/PaddleClas/docs/zh_CN/others/update_history.md b/src/PaddleClas/docs/zh_CN/others/update_history.md new file mode 100644 index 0000000..55ad1a2 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/update_history.md @@ -0,0 +1,60 @@ +# 更新日志 + +- 2021.11.1 发布[PP-ShiTu技术报告](https://arxiv.org/pdf/2111.00775.pdf)，新增饮料识别demo。 +- 2021.10.23 发布轻量级图像识别系统PP-ShiTu，CPU上0.2s即可完成在10w+库的图像识别。[点击这里](../quick_start/quick_start_recognition.md)立即体验。 +- 2021.09.17 发布PP-LCNet系列超轻量骨干网络模型, 在Intel CPU上，单张图像预测速度约5ms，ImageNet-1K数据集上Top1识别准确率达到80.82%，超越ResNet152的模型效果。PP-LCNet的介绍可以参考[论文](https://arxiv.org/pdf/2109.15099.pdf), 或者[PP-LCNet模型介绍](../models/PP-LCNet.md)，相关指标和预训练权重可以从 [这里](../algorithm_introduction/ImageNet_models.md)下载。 +- 2021.08.11 更新7个[FAQ](../faq_series/faq_2021_s2.md)。 +- 2021.06.29 添加Swin-transformer系列模型，ImageNet1k数据集上Top1 acc最高精度可达87.2%；支持训练预测评估与whl包部署，预训练模型可以从[这里](../models/models_intro.md)下载。 +- 2021.06.22,23,24 PaddleClas官方研发团队带来技术深入解读三日直播课。课程回放：[https://aistudio.baidu.com/aistudio/course/introduce/24519](https://aistudio.baidu.com/aistudio/course/introduce/24519) +- 2021.06.16 PaddleClas v2.2版本升级，集成Metric learning，向量检索等组件。新增商品识别、动漫人物识别、车辆识别和logo识别等4个图像识别应用。新增LeViT、Twins、TNT、DLA、HarDNet、RedNet系列30个预训练模型。 +- 2021.08.11 更新 7 个[FAQ](../faq_series/faq_2021_s2.md)。 +- 2021.06.29 添加 Swin-transformer 系列模型，ImageNet1k 数据集上 Top1 acc 最高精度可达 87.2%；支持训练预测评估与 whl 包部署，预训练模型可以从[这里](../models/models_intro.md)下载。 +- 2021.06.22,23,24 PaddleClas 官方研发团队带来技术深入解读三日直播课。课程回放：[https://aistudio.baidu.com/aistudio/course/introduce/24519](https://aistudio.baidu.com/aistudio/course/introduce/24519) +- 2021.06.16 PaddleClas v2.2 版本升级，集成 Metric learning，向量检索等组件。新增商品识别、动漫人物识别、车辆识别和 logo 识别等 4 个图像识别应用。新增 LeViT、Twins、TNT、DLA、HarDNet、RedNet 系列 30 个预训练模型。 +- 2021.04.15 + - 添加 `MixNet_L` 和 `ReXNet_3_0` 系列模型，在 ImageNet-1k 上 `MixNet` 模型 Top1 Acc 可达 78.6%，`ReXNet` 模型可达 82.09% +- 2021.01.27 + * 添加 ViT 与 DeiT 模型，在 ImageNet 上，ViT 模型 Top-1 Acc 可达 81.05%，DeiT 模型可达 85.5%。 +- 2021.01.08 + * 添加 whl 包及其使用说明，直接安装 paddleclas whl 包，即可快速完成模型预测。 +- 2020.12.16 + * 添加对 cpp 预测的 tensorRT 支持，预测加速更明显。 +- 2020.12.06 + * 添加 SE_HRNet_W64_C_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.8475。 +- 2020.11.23 + * 添加 GhostNet_x1_3_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.7938。 +- 2020.11.09 + * 添加 InceptionV3 结构和模型，在 ImageNet 上 Top-1 Acc 可达 0.791。 +- 2020.10.20 + * 添加 Res2Net50_vd_26w_4s_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.831；添加 Res2Net101_vd_26w_4s_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.839。 +- 2020.10.12 + * 添加 Paddle-Lite demo。 +- 2020.10.10 + * 添加 cpp inference demo。 + * 添加 FAQ 30 问。 +- 2020.09.17 + * 添加 HRNet_W48_C_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.836；添加 ResNet34_vd_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.797。 + +* 2020.09.07 + * 添加 HRNet_W18_C_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.81162；添加 MobileNetV3_small_x0_35_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 0.5555。 + +* 2020.07.14 + * 添加 Res2Net200_vd_26w_4s_ssld 模型，在 ImageNet 上 Top-1 Acc 可达 85.13%。 + * 添加 Fix_ResNet50_vd_ssld_v2 模型，，在 ImageNet 上 Top-1 Acc 可达 84.0%。 + +* 2020.06.17 + * 添加英文文档。 + +* 2020.06.12 + * 添加对 windows 和 CPU 环境的训练与评估支持。 + +* 2020.05.17 + * 添加混合精度训练。 + +* 2020.05.09 + * 添加 Paddle Serving 使用文档。 + * 添加 Paddle-Lite 使用文档。 + * 添加 T4 GPU 的 FP32/FP16 预测速度 benchmark。 + +* 2020.04.10: + * 第一次提交。 diff --git a/src/PaddleClas/docs/zh_CN/others/versions.md b/src/PaddleClas/docs/zh_CN/others/versions.md new file mode 100644 index 0000000..7399bdc --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/others/versions.md @@ -0,0 +1,58 @@ +# 版本更新信息 +---------- +## 目录 +* [1. v2.3](#1) +* [2. v2.2](#2) + + + +## 1. v2.3 + +- 模型更新 + - 添加轻量化模型预训练权重，包括检测模型、特征模型 + - 发布 PP-LCNet 系列模型，此系列模型是专门在 CPU 上设计运行的自研模型 + - SwinTransformer、Twins、Deit 支持从 scrach 直接训练，达到论文精度 +- 框架基础能力 + - 添加 DeepHash 模块，支持特征模型直接输出二值特征 + - 添加 PKSampler，特征模型不能多机多卡的训练的问题 + - 支持 PaddleSlim：支持分类模型、特征模型的量化、裁剪训练及离线量化功能 + - Legendary models 支持模型中间结果输出 + - 支持多标签分类训练 +- 预测部署 + - 使用 Faiss 替换原有特征检索库，提升平台适配性 + - 支持 PaddleServing：支持分类模型、图像识别流程的部署 + +- 推荐库版本 + - python 版本：3.7 + - PaddlePaddle 版本：2.1.3 + - PaddleSlim 版本：2.2.0 + - PaddleServing 版本：0.6.1 + + + +## 2. v2.2 + +- 模型更新 + - 添加 LeViT、Twins、TNT、DLA、HardNet、RedNet、SwinTransfomer 模型 +- 框架基础能力 + - 将分类模型分为两类 + - legendary models：引入 TheseusLayer 基类，及增加了修改网络功能接口，同时支持网络截断输出功能 + - model zoo：其他普通分类模型 + - 添加 Metric Learning 算法支持 + - 添加多种相关 Loss 算法，及基础网络模块 gears（支持与 backbone、loss 组合）方便使用 + - 同时支持普通分类及 metric learning 相关任务训练 + - 支持静态图训练 + - 分类训练支持 dali 加速 + - 支持 fp16 训练 +- 应用更新 + - 添加商品识别、车辆识别（车辆细粒度分类、车辆 ReID）、logo 识别、动漫人物识别应用具体案例及相关模型 + - 添加图像识别完整 pipeline，包含检测模块、特征提取模块、向量检索模块 +- 预测部署 + - 添加百度自研向量检索模块 Mobius，支持图像识别系统预测部署 + - 图像识别，建立特征库支持 batch_size>1 +- 文档更新 + - 添加图像识别相关文档 + - 修复之前文档 bug +- 推荐库版本 + - python 版本：3.7 + - PaddlePaddle：2.1.2 diff --git a/src/PaddleClas/docs/zh_CN/quick_start/index.rst b/src/PaddleClas/docs/zh_CN/quick_start/index.rst new file mode 100644 index 0000000..50af471 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/quick_start/index.rst @@ -0,0 +1,10 @@ +快速开始 +================================ + +.. toctree:: + :maxdepth: 2 + + quick_start_classification_new_user.md + quick_start_classification_professional.md + quick_start_recognition.md + quick_start_multilabel_classification.md diff --git a/src/PaddleClas/docs/zh_CN/quick_start/quick_start_classification_new_user.md b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_classification_new_user.md new file mode 100644 index 0000000..905f62d --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_classification_new_user.md @@ -0,0 +1,198 @@ +# 30分钟玩转PaddleClas（尝鲜版） + +此教程主要针对初级用户，即深度学习相关理论知识处于入门阶段，具有一定的 Python 基础，能够阅读简单代码的用户。此内容主要包括使用 PaddleClas 进行图像分类网络训练及模型预测。 + +--- +## 目录 +* [1. 基础知识](#1) +* [2. 环境安装与配置](#2) +* [3. 数据的准备与处理](#3) +* [4. 模型训练](#4) + * [4.1 使用CPU进行模型训练](#4.1) + * [4.1.1 不使用预训练模型](#4.1.1) + * [4.1.2 使用预训练模型](#4.1.2) + * [4.2 使用GPU进行模型训练](#4.2) + * [4.2.1 不使用预训练模型](#4.2.1) + * [4.2.2 使用预训练模型进行训练](#4.2.2) +* [5. 模型预测](#5) + + + +## 1. 基础知识 + +图像分类顾名思义就是一个模式分类问题，是计算机视觉中最基础的任务，它的目标是将不同的图像，划分到不同的类别。以下会对整个模型训练过程中需要了解到的一些概念做简单的解释，希望能够对初次体验 PaddleClas 的你有所帮助： + +- train/val/test dataset 分别代表模型的训练集、验证集和测试集： + - 训练集（train dataset）：用来训练模型，使模型能够识别不同类型的特征； + - 验证集（val dataset）：训练过程中的测试集，方便训练过程中查看模型训练程度； + - 测试集（test dataset）：训练模型结束后，用于评价模型结果的测试集。 + +- 预训练模型 + + 使用在某个较大的数据集训练好的预训练模型，即被预置了参数的权重，可以帮助模型在新的数据集上更快收敛。尤其是对一些训练数据比较稀缺的任务，在神经网络参数十分庞大的情况下，仅仅依靠任务自身的训练数据可能无法训练充分，加载预训练模型的方法可以认为是让模型基于一个更好的初始状态进行学习，从而能够达到更好的性能。 + +- 迭代轮数（epoch） + + 模型训练迭代的总轮数，模型对训练集全部样本过一遍即为一个 epoch。当测试错误率和训练错误率相差较小时，可认为当前迭代轮数合适；当测试错误率先变小后变大时，则说明迭代轮数过大，需要减小迭代轮数，否则容易出现过拟合。 + +- 损失函数（Loss Function） + + 训练过程中，衡量模型输出（预测值）与真实值之间的差异 + +- 准确率（Acc）：表示预测正确的样本数占总数据的比例 + + - Top1 Acc：预测结果中概率最大的所在分类正确，则判定为正确； + - Top5 Acc：预测结果中概率排名前 5 中有分类正确，则判定为正确； + + + +## 2. 环境安装与配置 + +具体安装步骤可详看[Paddle 安装文档](../installation/install_paddle.md)，[PaddleClas 安装文档](../installation/install_paddleclas.md)。 + + + +## 3. 数据的准备与处理 + +进入PaddleClas目录： + +```shell +# linux or mac， $path_to_PaddleClas表示PaddleClas的根目录，用户需要根据自己的真实目录修改 +cd $path_to_PaddleClas +``` + +进入 `dataset/flowers102` 目录，下载并解压 flowers102 数据集： + +```shell +# linux or mac +cd dataset/ +# 如果希望从浏览器中直接下载，可以复制该链接并访问，然后下载解压即可 +wget https://paddle-imagenet-models-name.bj.bcebos.com/data/flowers102.zip +# 解压 +unzip flowers102.zip +``` + +没有安装 `wget` 命令或者在 Windows 中下载的话，需要将地址拷贝到浏览器中下载，并进行解压到目录 `PaddleClas/dataset/` 下面即可。 + +解压完成后，在目录 `PaddleClas/dataset/flowers102` 下有用于训练和测试的三个 `.txt` 文件：`train_list.txt`（训练集，1020张图）、`val_list.txt`（验证集，1020张图）、`train_extra_list.txt`（更大的训练集，7169张图）。文件中每行格式：**图像相对路径** **图像的label_id**（注意：中间有空格），此外还有flowers102数据集 label id 与类别名称的映射文件：`flowers102_label_list.txt`。 + +flowers102数据集的图像文件存放在 `dataset/flowers102/jpg` 目录中，图像示例如下： + +![](../../images/quick_start/Examples-Flower-102.png) + +返回 `PaddleClas` 根目录： + +```shell +# linux or mac +cd ../../ +# windoes直接打开PaddleClas根目录即可 +``` + + +## 4. 模型训练 + + + +### 4.1 使用CPU进行模型训练 + +由于使用CPU来进行模型训练，计算速度较慢，因此，此处以 ShuffleNetV2_x0_25 为例。此模型计算量较小，在 CPU 上计算速度较快。但是也因为模型较小，训练好的模型精度也不会太高。 + + + +#### 4.1.1 不使用预训练模型 + +```shell +# windows在cmd中进入PaddleClas根目录，执行此命令 +python tools/train.py -c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml +``` + +- `-c` 参数是指定训练的配置文件路径，训练的具体超参数可查看`yaml`文件 +- `yaml`文`Global.device` 参数设置为`cpu`，即使用CPU进行训练（若不设置，此参数默认为`True`） +- `yaml`文件中`epochs`参数设置为20，说明对整个数据集进行20个epoch迭代，预计训练20分钟左右（不同CPU，训练时间略有不同），此时训练模型不充分。若提高训练模型精度，请将此参数设大，如**40**，训练时间也会相应延长 + + + +#### 4.1.2 使用预训练模型 + +```shell +python tools/train.py -c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml -o Arch.pretrained=True +``` + +- `-o` 参数可以选择为 `True` 或 `False`，也可以是预训练模型存放路径，当选择为 `True` 时，预训练权重会自动下载到本地。注意：若为预训练模型路径，则不要加上：`.pdparams` + +可以使用将使用与不使用预训练模型训练进行对比，观察 loss 的下降情况。 + + + +### 4.2 使用GPU进行模型训练 + +由于 GPU 训练速度更快，可以使用更复杂模型，因此以 ResNet50_vd 为例。与 ShuffleNetV2_x0_25 相比，此模型计算量较大，训练好的模型精度也会更高。 + +首先要设置环境变量，使用 0 号 GPU 进行训练： + +- 对于 Linux 用户 + + ```shell + export CUDA_VISIBLE_DEVICES=0 + ``` + +- 对于 Windows 用户 + + ```shell + set CUDA_VISIBLE_DEVICES=0 + ``` + + +#### 4.2.1. 不使用预训练模型 + +```shell +python tools/train.py -c ./ppcls/configs/quick_start/ResNet50_vd.yaml +``` + +训练完成后，验证集的`Top1 Acc`曲线如下所示，最高准确率为0.2735。训练精度曲线下图所示 + +![](../../images/quick_start/r50_vd_acc.png) + + + +#### 4.2.2 使用预训练模型进行训练 + +基于 ImageNet1k 分类预训练模型进行微调，训练脚本如下所示 + +```shell +python tools/train.py -c ./ppcls/configs/quick_start/ResNet50_vd.yaml -o Arch.pretrained=True +``` + +**注**：此训练脚本使用 GPU，如使用 CPU 可按照上文中[4.1 使用CPU进行模型训练](#4.1)所示，进行修改。 + +验证集的 `Top1 Acc` 曲线如下所示，最高准确率为 `0.9402`，加载预训练模型之后，flowers102 数据集精度大幅提升，绝对精度涨幅超过 65%。 + +![](../../images/quick_start/r50_vd_pretrained_acc.png) + + + +## 5. 模型预测 + +训练完成后，可以使用训练好的模型进行预测，以训练的 ResNet50_vd 模型为例，预测代码如下： + +```shell +cd $path_to_PaddleClas +python tools/infer.py -c ./ppcls/configs/quick_start/ResNet50_vd.yaml -o Infer.infer_imgs=dataset/flowers102/jpg/image_00001.jpg -o Global.pretrained_model=output/ResNet50_vd/best_model +``` + +`-i` 输入为单张图像路径，运行成功后，示例结果如下： + +`[{'class_ids': [76, 51, 37, 33, 9], 'scores': [0.99998, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00001.jpg', 'label_names': ['passion flower', 'wild pansy', 'great masterwort', 'mexican aster', 'globe thistle']}]` + +当然也可以使用训练的 ShuffleNetV2_x0_25 模型进行预测，代码如下： + +```shell +cd $path_to_PaddleClas +python tools/infer.py -c ./ppcls/configs/quick_start/new_user/ShuffleNetV2_x0_25.yaml -o Infer.infer_imgs=dataset/flowers102/jpg/image_00001.jpg -o Global.pretrained_model=output/ShuffleNetV2_x0_25/best_model +``` + +`-i` 参数也可以为待测图像文件所在目录（`dataset/flowers102/jpg/`），运行成功后，部分示例结果如下： + +`[{'class_ids': [76, 51, 37, 33, 9], 'scores': [0.99998, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00001.jpg', 'label_names': ['passion flower', 'wild pansy', 'great masterwort', 'mexican aster', 'globe thistle']}, {'class_ids': [76, 51, 37, 33, 32], 'scores': [0.99999, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00002.jpg', 'label_names': ['passion flower', 'wild pansy', 'great masterwort', 'mexican aster', 'love in the mist']}, {'class_ids': [76, 12, 39, 73, 78], 'scores': [0.99998, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00003.jpg', 'label_names': ['passion flower', 'king protea', 'lenten rose', 'rose', 'toad lily']}, {'class_ids': [76, 37, 34, 12, 9], 'scores': [0.86282, 0.11177, 0.00717, 0.00599, 0.00397], 'file_name': 'dataset/flowers102/jpg/image_00004.jpg', 'label_names': ['passion flower', 'great masterwort', 'alpine sea holly', 'king protea', 'globe thistle']}, {'class_ids': [76, 37, 33, 51, 69], 'scores': [0.9999, 1e-05, 1e-05, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00005.jpg', 'label_names': ['passion flower', 'great masterwort', 'mexican aster', 'wild pansy', 'tree poppy']}, {'class_ids': [76, 37, 51, 33, 73], 'scores': [0.99999, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00006.jpg', 'label_names': ['passion flower', 'great masterwort', 'wild pansy', 'mexican aster', 'rose']}, {'class_ids': [76, 37, 12, 91, 30], 'scores': [0.98746, 0.00211, 0.00201, 0.00136, 0.0007], 'file_name': 'dataset/flowers102/jpg/image_00007.jpg', 'label_names': ['passion flower', 'great masterwort', 'king protea', 'bee balm', 'carnation']}, {'class_ids': [76, 37, 81, 77, 72], 'scores': [0.99976, 3e-05, 2e-05, 2e-05, 1e-05], 'file_name': 'dataset/flowers102/jpg/image_00008.jpg', 'label_names': ['passion flower', 'great masterwort', 'clematis', 'lotus', 'water lily']}, {'class_ids': [76, 37, 13, 12, 34], 'scores': [0.99646, 0.00144, 0.00092, 0.00035, 0.00027], 'file_name': 'dataset/flowers102/jpg/image_00009.jpg', 'label_names': ['passion flower', 'great masterwort', 'spear thistle', 'king protea', 'alpine sea holly']}, {'class_ids': [76, 37, 34, 33, 51], 'scores': [0.99999, 0.0, 0.0, 0.0, 0.0], 'file_name': 'dataset/flowers102/jpg/image_00010.jpg', 'label_names': ['passion flower', 'great masterwort', 'alpine sea holly', 'mexican aster', 'wild pansy']}]` + +其中，列表的长度为 batch_size 的大小。 diff --git a/src/PaddleClas/docs/zh_CN/quick_start/quick_start_classification_professional.md b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_classification_professional.md new file mode 100644 index 0000000..5a13041 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_classification_professional.md @@ -0,0 +1,305 @@ +# 30 分钟玩转 PaddleClas（进阶版） + +此处提供了专业用户在 linux 操作系统上使用 PaddleClas 的快速上手教程，主要内容基于 CIFAR-100 数据集，快速体验不同模型的训练、加载不同预训练模型、SSLD 知识蒸馏方案和数据增广的效果。请事先参考[安装指南](../installation/install_paddleclas.md)配置运行环境和克隆 PaddleClas 代码。 + +------ + +## 目录 + +- [1. 数据和模型准备](#1) + - [1.1 数据准备](#1.1) + - [1.1.1 准备 CIFAR100](#1.1.1) +- [2. 模型训练](#2) + - [2.1 单标签训练](#2.1) + - [2.1.1 零基础训练：不加载预训练模型的训练](#2.1.1) + - [2.1.2 迁移学习](#2.1.2) +- [3. 数据增广](#3) + - [3.1 数据增广的尝试-Mixup](#3.1) +- [4. 知识蒸馏](#4) +- [5. 模型评估与推理](#5) + - [5.1 单标签分类模型评估与推理](#5.1) + - [5.1.1 单标签分类模型评估](#5.1.1) + - [5.1.2 单标签分类模型预测](#5.1.2) + - [5.1.3 单标签分类使用 inference 模型进行模型推理](#5.1.3) + + + +## 1. 数据和模型准备 + + + +### 1.1 数据准备 + + +* 进入 PaddleClas 目录。 + +``` +cd path_to_PaddleClas +``` + + + +#### 1.1.1 准备 CIFAR100 + +* 进入 `dataset/` 目录，下载并解压 CIFAR100 数据集。 + +```shell +cd dataset +wget https://paddle-imagenet-models-name.bj.bcebos.com/data/CIFAR100.tar +tar -xf CIFAR100.tar +cd ../ +``` + + + +## 2. 模型训练 + + + +### 2.1 单标签训练 + + + +#### 2.1.1 零基础训练：不加载预训练模型的训练 + +* 基于 ResNet50_vd 模型，训练脚本如下所示。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" +``` + +验证集的最高准确率为 0.415 左右。 + + + + +#### 2.1.2 迁移学习 + +* 基于 ImageNet1k 分类预训练模型 ResNet50_vd_pretrained（准确率 79.12%）进行微调，训练脚本如下所示。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" \ + -o Arch.pretrained=True +``` + +验证集最高准确率为 0.718 左右，加载预训练模型之后，CIFAR100 数据集精度大幅提升，绝对精度涨幅 30%。 + +* 基于 ImageNet1k 分类预训练模型 ResNet50_vd_ssld_pretrained（准确率 82.39%）进行微调，训练脚本如下所示。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" \ + -o Arch.pretrained=True \ + -o Arch.use_ssld=True +``` + +最终 CIFAR100 验证集上精度指标为 0.73，相对于 79.12% 预训练模型的微调结构，新数据集指标可以再次提升 1.2%。 + +* 替换 backbone 为 MobileNetV3_large_x1_0 进行微调，训练脚本如下所示。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV3_large_x1_0_CIFAR100_finetune.yaml \ + -o Global.output_dir="output_CIFAR" \ + -o Arch.pretrained=True +``` + +验证集最高准确率为 0.601 左右, 较 ResNet50_vd 低近 12%。 + + + + +## 3. 数据增广 + +PaddleClas 包含了很多数据增广的方法，如 Mixup、Cutout、RandomErasing 等，具体的方法可以参考[数据增广的章节](../algorithm_introduction/DataAugmentation.md)。 + + + +### 3.1 数据增广的尝试-Mixup + +基于[数据增广的章节](../algorithm_introduction/DataAugmentation.md) `3.3 节` 中的训练方法，结合 Mixup 的数据增广方式进行训练，具体的训练脚本如下所示。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_mixup_CIFAR100_finetune.yaml \ + -o Global.output_dir="output_CIFAR" + +``` + +最终 CIFAR100 验证集上的精度为 0.73，使用数据增广可以使得模型精度再次提升约 1.2%。 + + + +* **注意** + + * 其他数据增广的配置文件可以参考 `ppcls/configs/ImageNet/DataAugment/` 中的配置文件。 +* 训练 CIFAR100 的迭代轮数较少，因此进行训练时，验证集的精度指标可能会有 1% 左右的波动。 + + + + +## 4. 知识蒸馏 + + +PaddleClas 包含了自研的 SSLD 知识蒸馏方案，具体的内容可以参考[知识蒸馏章节](../algorithm_introduction/knowledge_distillation.md), 本小节将尝试使用知识蒸馏技术对 MobileNetV3_large_x1_0 模型进行训练，使用 `2.1.2 小节` 训练得到的 ResNet50_vd 模型作为蒸馏所用的教师模型，首先将 `2.1.2 小节` 训练得到的 ResNet50_vd 模型保存到指定目录，脚本如下。 + +```shell +mkdir pretrained +cp -r output_CIFAR/ResNet50_vd/best_model.pdparams ./pretrained/ +``` + +配置文件中模型名字、教师模型和学生模型的配置、预训练地址配置以及 freeze_params 配置如下，其中 `freeze_params_list` 中的两个值分别代表教师模型和学生模型是否冻结参数训练。 + +```yaml +Arch: + name: "DistillationModel" + # if not null, its lengths should be same as models + pretrained_list: + # if not null, its lengths should be same as models + freeze_params_list: + - True + - False + models: + - Teacher: + name: ResNet50_vd + pretrained: "./pretrained/best_model" + - Student: + name: MobileNetV3_large_x1_0 + pretrained: True +``` + +Loss 配置如下，其中训练 Loss 是学生模型的输出和教师模型的输出的交叉熵、验证 Loss 是学生模型的输出和真实标签的交叉熵。 + +```yaml +Loss: + Train: + - DistillationCELoss: + weight: 1.0 + model_name_pairs: + - ["Student", "Teacher"] + Eval: + - DistillationGTCELoss: + weight: 1.0 + model_names: ["Student"] +``` + +最终的训练脚本如下所示。 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/R50_vd_distill_MV3_large_x1_0_CIFAR100.yaml \ + -o Global.output_dir="output_CIFAR" + +``` + +最终 CIFAR100 验证集上的精度为 64.4%，使用教师模型进行知识蒸馏，MobileNetV3 的精度涨幅 4.3%。 + +* **注意** + + * 蒸馏过程中，教师模型使用的预训练模型为 CIFAR100 数据集上的训练结果，学生模型使用的是 ImageNet1k 数据集上精度为 75.32% 的 MobileNetV3_large_x1_0 预训练模型。 + * 该蒸馏过程无须使用真实标签，所以可以使用更多的无标签数据，在使用过程中，可以将无标签数据生成假的 `train_list.txt`，然后与真实的 `train_list.txt` 进行合并, 用户可以根据自己的数据自行体验。 + + + +## 5. 模型评估与推理 + + + +### 5.1 单标签分类模型评估与推理 + + + +#### 5.1.1 单标签分类模型评估。 + +训练好模型之后，可以通过以下命令实现对模型精度的评估。 + +```bash +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.pretrained_model="output_CIFAR/ResNet50_vd/best_model" +``` + + + +#### 5.1.2 单标签分类模型预测 + +模型训练完成之后，可以加载训练得到的预训练模型，进行模型预测。在模型库的 `tools/infer.py` 中提供了完整的示例，只需执行下述命令即可完成模型预测： + +```python +python3 tools/infer.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Infer.infer_imgs=./dataset/CIFAR100/test/0/0001.png \ + -o Global.pretrained_model=output_CIFAR/ResNet50_vd/best_model +``` + + + +#### 5.1.3 单标签分类使用 inference 模型进行模型推理 + +通过导出 inference 模型，PaddlePaddle 支持使用预测引擎进行预测推理。接下来介绍如何用预测引擎进行推理： +首先，对训练好的模型进行转换： + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/professional/ResNet50_vd_CIFAR100.yaml \ + -o Global.pretrained_model=output_CIFAR/ResNet50_vd/best_model +``` + +* 默认会在 `inference` 文件夹下生成 `inference.pdiparams`、`inference.pdmodel` 和 `inference.pdiparams.info` 文件。 + +使用预测引擎进行推理： + +进入 deploy 目录下： + +```bash +cd deploy +``` + +更改 `inference_cls.yaml` 文件，由于训练 CIFAR100 采用的分辨率是 32x32，所以需要改变相关的分辨率，最终配置文件中的图像预处理如下： + +```yaml +PreProcess: + transform_ops: + - ResizeImage: + resize_short: 36 + - CropImage: + size: 32 + - NormalizeImage: + scale: 0.00392157 + mean: [0.485, 0.456, 0.406] + std: [0.229, 0.224, 0.225] + order: '' + - ToCHWImage: +``` + +执行命令进行预测，由于默认 `class_id_map_file` 是 ImageNet 数据集的映射文件，所以此处需要置 None。 + +```bash +python3 python/predict_cls.py \ + -c configs/inference_cls.yaml \ + -o Global.infer_imgs=../dataset/CIFAR100/test/0/0001.png \ + -o PostProcess.Topk.class_id_map_file=None +``` diff --git a/src/PaddleClas/docs/zh_CN/quick_start/quick_start_multilabel_classification.md b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_multilabel_classification.md new file mode 100644 index 0000000..888a615 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_multilabel_classification.md @@ -0,0 +1,109 @@ +# 多标签分类 quick start + +基于 [NUS-WIDE-SCENE](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html) 数据集，体验多标签分类的训练、评估、预测的过程，该数据集是 NUS-WIDE 数据集的一个子集。请首先安装 PaddlePaddle 和 PaddleClas，具体安装步骤可详看 [Paddle 安装文档](../installation/install_paddle.md)，[PaddleClas 安装文档](../installation/install_paddleclas.md)。 + + +## 目录 + +* [1. 数据和模型准备](#1) +* [2. 模型训练](#2) +* [3. 模型评估](#3) +* [4. 模型预测](#4) +* [5. 基于预测引擎预测](#5) + * [5.1 导出 inference model](#5.1) + * [5.2 基于预测引擎预测](#5.2) + + +## 1. 数据和模型准备 + +* 进入 `PaddleClas` 目录。 + +``` +cd path_to_PaddleClas +``` + +* 创建并进入 `dataset/NUS-WIDE-SCENE` 目录，下载并解压 NUS-WIDE-SCENE 数据集。 + +```shell +mkdir dataset/NUS-WIDE-SCENE +cd dataset/NUS-WIDE-SCENE +wget https://paddle-imagenet-models-name.bj.bcebos.com/data/NUS-SCENE-dataset.tar +tar -xf NUS-SCENE-dataset.tar +``` + +* 返回 `PaddleClas` 根目录 + +``` +cd ../../ +``` + + +## 2. 模型训练 + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml +``` + +训练 10 epoch 之后，验证集最好的正确率应该在 0.95 左右。 + + + +## 3. 模型评估 + +```bash +python3 tools/eval.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + + +## 4. 模型预测 + +```bash +python3 tools/infer.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` + +得到类似下面的输出： +``` +[{'class_ids': [6, 13, 17, 23, 26, 30], 'scores': [0.95683, 0.5567, 0.55211, 0.99088, 0.5943, 0.78767], 'file_name': './deploy/images/0517_2715693311.jpg', 'label_names': []}] +``` + + +## 5. 基于预测引擎预测 + + +### 5.1 导出 inference model + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/quick_start/professional/MobileNetV1_multilabel.yaml \ + -o Arch.pretrained="./output/MobileNetV1/best_model" +``` +inference model 的路径默认在当前路径下 `./inference` + + +### 5.2 基于预测引擎预测 + +首先进入 `deploy` 目录下： + +```bash +cd ./deploy +``` + +通过预测引擎推理预测： + +``` +python3 python/predict_cls.py \ + -c configs/inference_multilabel_cls.yaml +``` + +得到类似下面的输出： +``` +0517_2715693311.jpg: class id(s): [6, 13, 17, 23, 26, 30], score(s): [0.96, 0.56, 0.55, 0.99, 0.59, 0.79], label_name(s): [] +``` diff --git a/src/PaddleClas/docs/zh_CN/quick_start/quick_start_recognition.md b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_recognition.md new file mode 100644 index 0000000..e2e6b16 --- /dev/null +++ b/src/PaddleClas/docs/zh_CN/quick_start/quick_start_recognition.md @@ -0,0 +1,334 @@ +# 图像识别快速开始 + +本文档包含 3 个部分：环境配置、图像识别体验、未知类别的图像识别体验。 + +如果图像类别已经存在于图像索引库中，那么可以直接参考[图像识别体验](#图像识别体验)章节，完成图像识别过程；如果希望识别未知类别的图像，即图像类别之前不存在于索引库中，那么可以参考[未知类别的图像识别体验](#未知类别的图像识别体验)章节，完成建立索引并识别的过程。 + +## 目录 + +* [1. 环境配置](#环境配置) +* [2. 图像识别体验](#图像识别体验) + * [2.1 下载、解压 inference 模型与 demo 数据](#2.1) + * [2.2 瓶装饮料识别与检索](#瓶装饮料识别与检索) + * [2.2.1 识别单张图像](#识别单张图像) + * [2.2.2 基于文件夹的批量识别](#基于文件夹的批量识别) +* [3. 未知类别的图像识别体验](#未知类别的图像识别体验) + * [3.1 准备新的数据与标签](#准备新的数据与标签) + * [3.2 建立新的索引库](#建立新的索引库) + * [3.3 基于新的索引库的图像识别](#基于新的索引库的图像识别) +* [4. 服务端识别模型列表](#4) + + + +## 1. 环境配置 + +* 安装：请先参考 [Paddle 安装教程](../installation/install_paddle.md) 以及 [PaddleClas 安装教程](../installation/install_paddleclas.md) 配置 PaddleClas 运行环境。 + +* 进入 `deploy` 运行目录。本部分所有内容与命令均需要在 `deploy` 目录下运行，可以通过下面的命令进入 `deploy` 目录。 + + ``` + cd deploy + ``` + + + +## 2. 图像识别体验 + +轻量级通用主体检测模型与轻量级通用识别模型和配置文件下载方式如下表所示。 + + + +| 模型简介 | 推荐场景 | inference 模型 | 预测配置文件 | +| ------------ | ------------- | -------- | ------- | +| 轻量级通用主体检测模型 | 通用场景 |[tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar) [zip 格式文件下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.zip) | - | +| 轻量级通用识别模型 | 通用场景 | [tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar) [zip 格式文件下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.zip) | [inference_general.yaml](../../../deploy/configs/inference_general.yaml) | +| 轻量级通用识别二值模型 | 检索库很大， 存储受限场景 | [tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_binary_v1.0_infer.tar) [zip 格式文件下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_binary_v1.0_infer.zip)| [inference_general_binary.yaml](../../../deploy/configs/inference_general_binary.yaml) | + +注意：由于部分解压缩软件在解压上述 `tar` 格式文件时存在问题，建议非命令行用户下载 `zip` 格式文件并解压。`tar` 格式文件建议使用命令 `tar xf xxx.tar` 解压。 + +本章节 demo 数据下载地址如下: [瓶装饮料数据下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar)。 + + +如果希望体验服务端主体检测和各垂类方向的识别模型，可以参考[第4章](#4)。 + +**注意** + +1. windows 环境下如果没有安装 wget, 可以按照下面的步骤安装 wget 与 tar 命令，也可以在下载模型时将链接复制到浏览器中下载，并解压放置在相应目录下； linux 或者 macOS 用户可以右键点击，然后复制下载链接，即可通过 `wget` 命令下载。 +2. 如果 macOS 环境下没有安装 `wget` 命令，可以运行下面的命令进行安装。 + +```shell +# 安装 homebrew +ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"; +# 安装 wget +brew install wget +``` + +4. 如果希望在 windows 环境下安装 wget，可以参考：[链接](https://www.cnblogs.com/jeshy/p/10518062.html)；如果希望在 windows 环境中安装 tar 命令，可以参考：[链接](https://www.cnblogs.com/chooperman/p/14190107.html)。 + + +* 可以按照下面的命令下载并解压数据与模型 + +```shell +mkdir models +cd models +# 下载识别 inference 模型并解压 +wget {模型下载链接地址} && tar -xf {压缩包的名称} +cd .. + +# 下载 demo 数据并解压 +wget {数据下载链接地址} && tar -xf {压缩包的名称} +``` + + + +### 2.1 下载、解压 inference 模型与 demo 数据 + +下载 demo 数据集以及轻量级主体检测、识别模型，命令如下。 + +```shell +mkdir models +cd models +# 下载通用检测 inference 模型并解压 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar && tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar +# 下载识别 inference 模型并解压 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar && tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar + +cd ../ +# 下载 demo 数据并解压 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar && tar -xf drink_dataset_v1.0.tar +``` + +解压完毕后，`drink_dataset_v1.0/` 文件夹下应有如下文件结构： + +``` +├── drink_dataset_v1.0/ +│ ├── gallery/ +│ ├── index/ +│ ├── test_images/ +├── ... +``` + +其中 `gallery` 文件夹中存放的是用于构建索引库的原始图像，`index` 表示基于原始图像构建得到的索引库信息，`test_images` 文件夹中存放的是用于测试识别效果的图像列表。 + + +`models` 文件夹下应有如下文件结构： + +``` +├── general_PPLCNet_x2_5_lite_v1.0_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +``` + +**注意** + +如果使用服务端通用识别模型，Demo 数据需要重新提取特征、够建索引，方式如下： + +```shell +# 下面是使用下载的服务端商品识别模型进行索引库构建 +python3.7 python/build_gallery.py -c configs/inference_general.yaml -o Global.rec_inference_model_dir=./models/general_PPLCNet_x2_5_lite_v1.0_infer +``` + + + +### 2.2 瓶装饮料识别与检索 + +以瓶装饮料识别 demo 为例，展示识别与检索过程（如果希望尝试其他方向的识别与检索效果，在下载解压好对应的 demo 数据与模型之后，替换对应的配置文件即可完成预测）。 + +注意，此部分使用了 `faiss` 作为检索库，安装方法如下： + +```python +pip install faiss-cpu==1.7.1post2 +``` + +若使用时，不能正常引用，则 `uninstall` 之后，重新 `install`，尤其是 windows 下。 + + + +#### 2.2.1 识别单张图像 + +运行下面的命令，对图像 `./drink_dataset_v1.0/test_images/nongfu_spring.jpeg` 进行识别与检索 + +```shell +# 使用下面的命令使用 GPU 进行预测 +python3.7 python/predict_system.py -c configs/inference_general.yaml +# 使用下面的命令使用 CPU 进行预测 +python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.use_gpu=False +``` + +待检索图像如下所示。 + +![](../../images/recognition/drink_data_demo/test_images/nongfu_spring.jpeg) + + +最终输出结果如下。 + +``` +[{'bbox': [244, 49, 509, 964], 'rec_docs': '农夫山泉-饮用天然水', 'rec_scores': 0.7585664}] +``` + +其中 `bbox` 表示检测出的主体所在位置，`rec_docs` 表示索引库中与检测框最为相似的类别，`rec_scores` 表示对应的置信度。 + +检测的可视化结果也保存在 `output` 文件夹下，对于本张图像，识别结果可视化如下所示。 + +![](../../images/recognition/drink_data_demo/output/nongfu_spring.jpeg) + + + +#### 2.2.2 基于文件夹的批量识别 + +如果希望预测文件夹内的图像，可以直接修改配置文件中的 `Global.infer_imgs` 字段，也可以通过下面的 `-o` 参数修改对应的配置。 + +```shell +# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False +python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v1.0/test_images/" +``` + +终端中会输出该文件夹内所有图像的识别结果，如下所示。 + +``` +... +[{'bbox': [345, 95, 524, 586], 'rec_docs': '红牛-强化型', 'rec_scores': 0.80164653}] +Inference: 23.43583106994629 ms per batch image +[{'bbox': [233, 0, 372, 436], 'rec_docs': '康师傅矿物质水', 'rec_scores': 0.72513914}] +Inference: 117.95639991760254 ms per batch image +[{'bbox': [138, 40, 573, 1198], 'rec_docs': '乐虎功能饮料', 'rec_scores': 0.7855944}] +Inference: 22.172927856445312 ms per batch image +[{'bbox': [328, 7, 467, 272], 'rec_docs': '脉动', 'rec_scores': 0.5829516}] +Inference: 118.08514595031738 ms per batch image +[{'bbox': [242, 82, 498, 726], 'rec_docs': '味全_每日C', 'rec_scores': 0.75581443}] +Inference: 150.06470680236816 ms per batch image +[{'bbox': [437, 71, 660, 728], 'rec_docs': '元气森林', 'rec_scores': 0.8478892}, {'bbox': [221, 72, 449, 701], 'rec_docs': '元气森林', 'rec_scores': 0.6790612}, {'bbox': [794, 104, 979, 652], 'rec_docs': '元气森林', 'rec_scores': 0.6292581}] +... +``` + +所有图像的识别结果可视化图像也保存在 `output` 文件夹内。 + + +更多地，可以通过修改 `Global.rec_inference_model_dir` 字段来更改识别 inference 模型的路径，通过修改 `IndexProcess.index_dir` 字段来更改索引库索引的路径。 + + + +## 3. 未知类别的图像识别体验 + +对图像 `./drink_dataset_v1.0/test_images/mosilian.jpeg` 进行识别，命令如下 + +```shell +# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False +python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v1.0/test_images/mosilian.jpeg" +``` + +待检索图像如下所示。 + +![](../../images/recognition/drink_data_demo/test_images/mosilian.jpeg) + + +输出结果为空。 + +由于默认的索引库中不包含对应的索引信息，所以这里的识别结果有误，此时我们可以通过构建新的索引库的方式，完成未知类别的图像识别。 + +当索引库中的图像无法覆盖我们实际识别的场景时，即在预测未知类别的图像时，我们需要将对应类别的相似图像添加到索引库中，从而完成对未知类别的图像识别，这一过程是不需要重新训练的。 + + + +### 3.1 准备新的数据与标签 + +首先需要将与待检索图像相似的图像列表拷贝到索引库原始图像的文件夹。这里 PaddleClas 已经将所有的图像数据都放在文件夹 `drink_dataset_v1.0/gallery/` 中。 + +然后需要编辑记录了图像路径和标签信息的文本文件，这里 PaddleClas 将更正后的标签信息文件放在了 `drink_dataset_v1.0/gallery/drink_label_all.txt` 文件中。可以与默认的 `drink_dataset_v1.0/gallery/drink_label.txt` 标签文件进行对比，添加了光明和三元系列牛奶的索引图像。 + + +每一行的文本中，第一个字段表示图像的相对路径，第二个字段表示图像对应的标签信息，中间用 `\t` 键分隔开（注意：有些编辑器会将 `tab` 自动转换为 `空格`，这种情况下会导致文件解析报错）。 + + + +### 3.2 建立新的索引库 + +使用下面的命令构建 `index` 索引，加速识别后的检索过程。 + +```shell +python3.7 python/build_gallery.py -c configs/inference_general.yaml -o IndexProcess.data_file="./drink_dataset_v1.0/gallery/drink_label_all.txt" -o IndexProcess.index_dir="./drink_dataset_v1.0/index_all" +``` + +最终新的索引信息保存在文件夹 `./drink_dataset_v1.0/index_all` 中。具体 `yaml` 请参考[向量检索文档](../image_recognition_pipeline/vector_search.md)。 + + + +### 3.3 基于新的索引库的图像识别 + +使用新的索引库，对上述图像进行识别，运行命令如下。 + +```shell +# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False +python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="././drink_dataset_v1.0/test_images/mosilian.jpeg" -o IndexProcess.index_dir="./drink_dataset_v1.0/index_all" +``` + +输出结果如下。 + +``` +[{'bbox': [396, 553, 508, 621], 'rec_docs': '光明_莫斯利安', 'rec_scores': 0.5921005}] +``` + +最终识别结果为`光明_莫斯利安`，识别正确，识别结果可视化如下所示。 + +![](../../images/recognition/drink_data_demo/output/mosilian.jpeg) + + + +## 4. 服务端识别模型列表 + +目前，我们更推荐您使用[轻量级通用主体检测模型与轻量级通用识别模型](#轻量级通用主体检测模型与轻量级通用识别模型)，以获得更好的测试结果。但是如果您希望体验服务端识别模型，服务器端通用主体检测模型与各方向识别模型、测试数据下载地址以及对应的配置文件地址如下。 + +| 模型简介 | 推荐场景 | inference 模型 | 预测配置文件 | +| ------------ | ------------- | -------- | ------- | +| 通用主体检测模型 | 通用场景 |[模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar) | - | +| Logo 识别模型 | Logo 场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/logo_rec_ResNet50_Logo3K_v1.0_infer.tar) | [inference_logo.yaml](../../../deploy/configs/inference_logo.yaml) | +| 动漫人物识别模型 | 动漫人物场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/cartoon_rec_ResNet50_iCartoon_v1.0_infer.tar) | [inference_cartoon.yaml](../../../deploy/configs/inference_cartoon.yaml) | +| 车辆细分类模型 | 车辆场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_cls_ResNet50_CompCars_v1.0_infer.tar) | [inference_vehicle.yaml](../../../deploy/configs/inference_vehicle.yaml) | +| 商品识别模型 | 商品场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar) | [inference_product.yaml](../../../deploy/configs/inference_product.yaml) | +| 车辆 ReID 模型 | 车辆 ReID 场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_reid_ResNet50_VERIWild_v1.0_infer.tar) | [inference_vehicle.yaml](../../../deploy/configs/inference_vehicle.yaml) | + +```shell +cd PaddleClas/deploy/ +mkdir -p models +``` + +```shell +cd ./models +# 下载通用主体检测模型并解压 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar && tar -xf ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar +# 下载识别模型并解压 +wget {识别模型下载链接地址} && tar -xf {压缩包的名称} +``` + +使用如下命令下载各方向识别模型的测试数据： + +```shell +# 回到 deploy 目录下 +cd .. +# 下载测试数据并解压 +wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognition_demo_data_en_v1.1.tar && tar -xf recognition_demo_data_en_v1.1.tar +``` + +解压完毕后，`recognition_demo_data_v1.1` 文件夹下应有如下文件结构： + +``` +├── recognition_demo_data_v1.1 +│ ├── gallery_cartoon +│ ├── gallery_logo +│ ├── gallery_product +│ ├── gallery_vehicle +│ ├── test_cartoon +│ ├── test_logo +│ ├── test_product +│ └── test_vehicle +├── ... +``` + +按照上述步骤下载模型和测试数据后，您可以进行相关方向识别模型的测试。 + +* 更多关于主体检测的介绍可以参考：[主体检测教程文档](../image_recognition_pipeline/mainbody_detection.md)；关于特征提取的介绍可以参考：[特征提取教程文档](../image_recognition_pipeline/feature_extraction.md)；关于向量检索的介绍可以参考：[向量检索教程文档](../image_recognition_pipeline/vector_search.md)。 diff --git a/src/PaddleClas/output/ResNet50_vd/infer.log b/src/PaddleClas/output/ResNet50_vd/infer.log new file mode 100644 index 0000000..2a4db87 --- /dev/null +++ b/src/PaddleClas/output/ResNet50_vd/infer.log @@ -0,0 +1,124 @@ +[2022/04/18 10:25:05] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:25:05] root INFO: Arch : +[2022/04/18 10:25:05] root INFO: class_num : 102 +[2022/04/18 10:25:05] root INFO: name : ResNet50_vd +[2022/04/18 10:25:05] root INFO: DataLoader : +[2022/04/18 10:25:05] root INFO: Eval : +[2022/04/18 10:25:05] root INFO: dataset : +[2022/04/18 10:25:05] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:25:05] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:25:05] root INFO: name : ImageNetDataset +[2022/04/18 10:25:05] root INFO: transform_ops : +[2022/04/18 10:25:05] root INFO: DecodeImage : +[2022/04/18 10:25:05] root INFO: channel_first : False +[2022/04/18 10:25:05] root INFO: to_rgb : True +[2022/04/18 10:25:05] root INFO: ResizeImage : +[2022/04/18 10:25:05] root INFO: resize_short : 256 +[2022/04/18 10:25:05] root INFO: CropImage : +[2022/04/18 10:25:05] root INFO: size : 224 +[2022/04/18 10:25:05] root INFO: NormalizeImage : +[2022/04/18 10:25:05] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:25:05] root INFO: order : +[2022/04/18 10:25:05] root INFO: scale : 1.0/255.0 +[2022/04/18 10:25:05] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:25:05] root INFO: loader : +[2022/04/18 10:25:05] root INFO: num_workers : 4 +[2022/04/18 10:25:05] root INFO: use_shared_memory : True +[2022/04/18 10:25:05] root INFO: sampler : +[2022/04/18 10:25:05] root INFO: batch_size : 64 +[2022/04/18 10:25:05] root INFO: drop_last : False +[2022/04/18 10:25:05] root INFO: name : DistributedBatchSampler +[2022/04/18 10:25:05] root INFO: shuffle : False +[2022/04/18 10:25:05] root INFO: Train : +[2022/04/18 10:25:05] root INFO: dataset : +[2022/04/18 10:25:05] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:25:05] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:25:05] root INFO: name : ImageNetDataset +[2022/04/18 10:25:05] root INFO: transform_ops : +[2022/04/18 10:25:05] root INFO: DecodeImage : +[2022/04/18 10:25:05] root INFO: channel_first : False +[2022/04/18 10:25:05] root INFO: to_rgb : True +[2022/04/18 10:25:05] root INFO: RandCropImage : +[2022/04/18 10:25:05] root INFO: size : 224 +[2022/04/18 10:25:05] root INFO: RandFlipImage : +[2022/04/18 10:25:05] root INFO: flip_code : 1 +[2022/04/18 10:25:05] root INFO: NormalizeImage : +[2022/04/18 10:25:05] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:25:05] root INFO: order : +[2022/04/18 10:25:05] root INFO: scale : 1.0/255.0 +[2022/04/18 10:25:05] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:25:05] root INFO: loader : +[2022/04/18 10:25:05] root INFO: num_workers : 4 +[2022/04/18 10:25:05] root INFO: use_shared_memory : True +[2022/04/18 10:25:05] root INFO: sampler : +[2022/04/18 10:25:05] root INFO: batch_size : 32 +[2022/04/18 10:25:05] root INFO: drop_last : False +[2022/04/18 10:25:05] root INFO: name : DistributedBatchSampler +[2022/04/18 10:25:05] root INFO: shuffle : True +[2022/04/18 10:25:05] root INFO: Global : +[2022/04/18 10:25:05] root INFO: checkpoints : None +[2022/04/18 10:25:05] root INFO: device : gpu +[2022/04/18 10:25:05] root INFO: epochs : 20 +[2022/04/18 10:25:05] root INFO: eval_during_train : True +[2022/04/18 10:25:05] root INFO: eval_interval : 1 +[2022/04/18 10:25:05] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:25:05] root INFO: output_dir : ./output/ +[2022/04/18 10:25:05] root INFO: pretrained_model : output/ResNet50_vd/best_model +[2022/04/18 10:25:05] root INFO: print_batch_step : 10 +[2022/04/18 10:25:05] root INFO: save_inference_dir : ./inference +[2022/04/18 10:25:05] root INFO: save_interval : 1 +[2022/04/18 10:25:05] root INFO: use_visualdl : False +[2022/04/18 10:25:05] root INFO: Infer : +[2022/04/18 10:25:05] root INFO: PostProcess : +[2022/04/18 10:25:05] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:25:05] root INFO: name : Topk +[2022/04/18 10:25:05] root INFO: topk : 5 +[2022/04/18 10:25:05] root INFO: batch_size : 10 +[2022/04/18 10:25:05] root INFO: infer_imgs : dataset/flowers102/jpg/image_00001.jpg +[2022/04/18 10:25:05] root INFO: transforms : +[2022/04/18 10:25:05] root INFO: DecodeImage : +[2022/04/18 10:25:05] root INFO: channel_first : False +[2022/04/18 10:25:05] root INFO: to_rgb : True +[2022/04/18 10:25:05] root INFO: ResizeImage : +[2022/04/18 10:25:05] root INFO: resize_short : 256 +[2022/04/18 10:25:05] root INFO: CropImage : +[2022/04/18 10:25:05] root INFO: size : 224 +[2022/04/18 10:25:05] root INFO: NormalizeImage : +[2022/04/18 10:25:05] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:25:05] root INFO: order : +[2022/04/18 10:25:05] root INFO: scale : 1.0/255.0 +[2022/04/18 10:25:05] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:25:05] root INFO: ToCHWImage : None +[2022/04/18 10:25:05] root INFO: Loss : +[2022/04/18 10:25:05] root INFO: Eval : +[2022/04/18 10:25:05] root INFO: CELoss : +[2022/04/18 10:25:05] root INFO: weight : 1.0 +[2022/04/18 10:25:05] root INFO: Train : +[2022/04/18 10:25:05] root INFO: CELoss : +[2022/04/18 10:25:05] root INFO: weight : 1.0 +[2022/04/18 10:25:05] root INFO: Metric : +[2022/04/18 10:25:05] root INFO: Eval : +[2022/04/18 10:25:05] root INFO: TopkAcc : +[2022/04/18 10:25:05] root INFO: topk : [1, 5] +[2022/04/18 10:25:05] root INFO: Train : +[2022/04/18 10:25:05] root INFO: TopkAcc : +[2022/04/18 10:25:05] root INFO: topk : [1, 5] +[2022/04/18 10:25:05] root INFO: Optimizer : +[2022/04/18 10:25:05] root INFO: lr : +[2022/04/18 10:25:05] root INFO: learning_rate : 0.0125 +[2022/04/18 10:25:05] root INFO: name : Cosine +[2022/04/18 10:25:05] root INFO: warmup_epoch : 5 +[2022/04/18 10:25:05] root INFO: momentum : 0.9 +[2022/04/18 10:25:05] root INFO: name : Momentum +[2022/04/18 10:25:05] root INFO: regularizer : +[2022/04/18 10:25:05] root INFO: coeff : 1e-05 +[2022/04/18 10:25:05] root INFO: name : L2 diff --git a/src/PaddleClas/output/ResNet50_vd/train.log b/src/PaddleClas/output/ResNet50_vd/train.log new file mode 100644 index 0000000..1fc4907 --- /dev/null +++ b/src/PaddleClas/output/ResNet50_vd/train.log @@ -0,0 +1,380 @@ +[2022/04/18 10:10:57] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:10:57] root INFO: Arch : +[2022/04/18 10:10:57] root INFO: class_num : 102 +[2022/04/18 10:10:57] root INFO: name : ResNet50_vd +[2022/04/18 10:10:57] root INFO: DataLoader : +[2022/04/18 10:10:57] root INFO: Eval : +[2022/04/18 10:10:57] root INFO: dataset : +[2022/04/18 10:10:57] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:10:57] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:10:57] root INFO: name : ImageNetDataset +[2022/04/18 10:10:57] root INFO: transform_ops : +[2022/04/18 10:10:58] root INFO: DecodeImage : +[2022/04/18 10:10:58] root INFO: channel_first : False +[2022/04/18 10:10:58] root INFO: to_rgb : True +[2022/04/18 10:10:58] root INFO: ResizeImage : +[2022/04/18 10:10:58] root INFO: resize_short : 256 +[2022/04/18 10:10:58] root INFO: CropImage : +[2022/04/18 10:10:58] root INFO: size : 224 +[2022/04/18 10:10:58] root INFO: NormalizeImage : +[2022/04/18 10:10:58] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:58] root INFO: order : +[2022/04/18 10:10:58] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:58] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:58] root INFO: loader : +[2022/04/18 10:10:58] root INFO: num_workers : 4 +[2022/04/18 10:10:58] root INFO: use_shared_memory : True +[2022/04/18 10:10:58] root INFO: sampler : +[2022/04/18 10:10:58] root INFO: batch_size : 64 +[2022/04/18 10:10:58] root INFO: drop_last : False +[2022/04/18 10:10:58] root INFO: name : DistributedBatchSampler +[2022/04/18 10:10:58] root INFO: shuffle : False +[2022/04/18 10:10:58] root INFO: Train : +[2022/04/18 10:10:58] root INFO: dataset : +[2022/04/18 10:10:58] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:10:58] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:10:58] root INFO: name : ImageNetDataset +[2022/04/18 10:10:58] root INFO: transform_ops : +[2022/04/18 10:10:58] root INFO: DecodeImage : +[2022/04/18 10:10:58] root INFO: channel_first : False +[2022/04/18 10:10:58] root INFO: to_rgb : True +[2022/04/18 10:10:58] root INFO: RandCropImage : +[2022/04/18 10:10:58] root INFO: size : 224 +[2022/04/18 10:10:58] root INFO: RandFlipImage : +[2022/04/18 10:10:58] root INFO: flip_code : 1 +[2022/04/18 10:10:58] root INFO: NormalizeImage : +[2022/04/18 10:10:58] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:58] root INFO: order : +[2022/04/18 10:10:58] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:58] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:58] root INFO: loader : +[2022/04/18 10:10:58] root INFO: num_workers : 4 +[2022/04/18 10:10:58] root INFO: use_shared_memory : True +[2022/04/18 10:10:58] root INFO: sampler : +[2022/04/18 10:10:58] root INFO: batch_size : 32 +[2022/04/18 10:10:58] root INFO: drop_last : False +[2022/04/18 10:10:58] root INFO: name : DistributedBatchSampler +[2022/04/18 10:10:58] root INFO: shuffle : True +[2022/04/18 10:10:58] root INFO: Global : +[2022/04/18 10:10:58] root INFO: checkpoints : None +[2022/04/18 10:10:58] root INFO: device : gpu +[2022/04/18 10:10:58] root INFO: epochs : 20 +[2022/04/18 10:10:58] root INFO: eval_during_train : True +[2022/04/18 10:10:58] root INFO: eval_interval : 1 +[2022/04/18 10:10:58] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:10:58] root INFO: output_dir : ./output/ +[2022/04/18 10:10:58] root INFO: pretrained_model : None +[2022/04/18 10:10:58] root INFO: print_batch_step : 10 +[2022/04/18 10:10:58] root INFO: save_inference_dir : ./inference +[2022/04/18 10:10:58] root INFO: save_interval : 1 +[2022/04/18 10:10:58] root INFO: use_visualdl : False +[2022/04/18 10:10:58] root INFO: Infer : +[2022/04/18 10:10:58] root INFO: PostProcess : +[2022/04/18 10:10:58] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:10:58] root INFO: name : Topk +[2022/04/18 10:10:58] root INFO: topk : 5 +[2022/04/18 10:10:58] root INFO: batch_size : 10 +[2022/04/18 10:10:58] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:10:58] root INFO: transforms : +[2022/04/18 10:10:58] root INFO: DecodeImage : +[2022/04/18 10:10:58] root INFO: channel_first : False +[2022/04/18 10:10:58] root INFO: to_rgb : True +[2022/04/18 10:10:58] root INFO: ResizeImage : +[2022/04/18 10:10:58] root INFO: resize_short : 256 +[2022/04/18 10:10:58] root INFO: CropImage : +[2022/04/18 10:10:58] root INFO: size : 224 +[2022/04/18 10:10:58] root INFO: NormalizeImage : +[2022/04/18 10:10:58] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:58] root INFO: order : +[2022/04/18 10:10:58] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:58] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:58] root INFO: ToCHWImage : None +[2022/04/18 10:10:58] root INFO: Loss : +[2022/04/18 10:10:58] root INFO: Eval : +[2022/04/18 10:10:58] root INFO: CELoss : +[2022/04/18 10:10:58] root INFO: weight : 1.0 +[2022/04/18 10:10:58] root INFO: Train : +[2022/04/18 10:10:58] root INFO: CELoss : +[2022/04/18 10:10:58] root INFO: weight : 1.0 +[2022/04/18 10:10:58] root INFO: Metric : +[2022/04/18 10:10:58] root INFO: Eval : +[2022/04/18 10:10:58] root INFO: TopkAcc : +[2022/04/18 10:10:58] root INFO: topk : [1, 5] +[2022/04/18 10:10:58] root INFO: Train : +[2022/04/18 10:10:58] root INFO: TopkAcc : +[2022/04/18 10:10:58] root INFO: topk : [1, 5] +[2022/04/18 10:10:58] root INFO: Optimizer : +[2022/04/18 10:10:58] root INFO: lr : +[2022/04/18 10:10:58] root INFO: learning_rate : 0.0125 +[2022/04/18 10:10:58] root INFO: name : Cosine +[2022/04/18 10:10:58] root INFO: warmup_epoch : 5 +[2022/04/18 10:10:58] root INFO: momentum : 0.9 +[2022/04/18 10:10:58] root INFO: name : Momentum +[2022/04/18 10:10:58] root INFO: regularizer : +[2022/04/18 10:10:58] root INFO: coeff : 1e-05 +[2022/04/18 10:10:58] root INFO: name : L2 +[2022/04/18 10:10:58] root INFO: profiler_options : None +[2022/04/18 10:11:26] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:11:26] root INFO: Arch : +[2022/04/18 10:11:26] root INFO: class_num : 102 +[2022/04/18 10:11:26] root INFO: name : ResNet50_vd +[2022/04/18 10:11:26] root INFO: pretrained : True +[2022/04/18 10:11:26] root INFO: DataLoader : +[2022/04/18 10:11:26] root INFO: Eval : +[2022/04/18 10:11:26] root INFO: dataset : +[2022/04/18 10:11:26] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:11:26] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:11:26] root INFO: name : ImageNetDataset +[2022/04/18 10:11:26] root INFO: transform_ops : +[2022/04/18 10:11:26] root INFO: DecodeImage : +[2022/04/18 10:11:26] root INFO: channel_first : False +[2022/04/18 10:11:26] root INFO: to_rgb : True +[2022/04/18 10:11:26] root INFO: ResizeImage : +[2022/04/18 10:11:26] root INFO: resize_short : 256 +[2022/04/18 10:11:26] root INFO: CropImage : +[2022/04/18 10:11:26] root INFO: size : 224 +[2022/04/18 10:11:26] root INFO: NormalizeImage : +[2022/04/18 10:11:26] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:11:26] root INFO: order : +[2022/04/18 10:11:26] root INFO: scale : 1.0/255.0 +[2022/04/18 10:11:26] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:11:26] root INFO: loader : +[2022/04/18 10:11:26] root INFO: num_workers : 4 +[2022/04/18 10:11:26] root INFO: use_shared_memory : True +[2022/04/18 10:11:26] root INFO: sampler : +[2022/04/18 10:11:26] root INFO: batch_size : 64 +[2022/04/18 10:11:26] root INFO: drop_last : False +[2022/04/18 10:11:26] root INFO: name : DistributedBatchSampler +[2022/04/18 10:11:26] root INFO: shuffle : False +[2022/04/18 10:11:26] root INFO: Train : +[2022/04/18 10:11:26] root INFO: dataset : +[2022/04/18 10:11:26] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:11:26] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:11:26] root INFO: name : ImageNetDataset +[2022/04/18 10:11:26] root INFO: transform_ops : +[2022/04/18 10:11:26] root INFO: DecodeImage : +[2022/04/18 10:11:26] root INFO: channel_first : False +[2022/04/18 10:11:26] root INFO: to_rgb : True +[2022/04/18 10:11:26] root INFO: RandCropImage : +[2022/04/18 10:11:26] root INFO: size : 224 +[2022/04/18 10:11:26] root INFO: RandFlipImage : +[2022/04/18 10:11:26] root INFO: flip_code : 1 +[2022/04/18 10:11:26] root INFO: NormalizeImage : +[2022/04/18 10:11:26] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:11:26] root INFO: order : +[2022/04/18 10:11:26] root INFO: scale : 1.0/255.0 +[2022/04/18 10:11:26] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:11:26] root INFO: loader : +[2022/04/18 10:11:26] root INFO: num_workers : 4 +[2022/04/18 10:11:26] root INFO: use_shared_memory : True +[2022/04/18 10:11:26] root INFO: sampler : +[2022/04/18 10:11:26] root INFO: batch_size : 32 +[2022/04/18 10:11:26] root INFO: drop_last : False +[2022/04/18 10:11:26] root INFO: name : DistributedBatchSampler +[2022/04/18 10:11:26] root INFO: shuffle : True +[2022/04/18 10:11:26] root INFO: Global : +[2022/04/18 10:11:26] root INFO: checkpoints : None +[2022/04/18 10:11:26] root INFO: device : gpu +[2022/04/18 10:11:26] root INFO: epochs : 20 +[2022/04/18 10:11:26] root INFO: eval_during_train : True +[2022/04/18 10:11:26] root INFO: eval_interval : 1 +[2022/04/18 10:11:26] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:11:26] root INFO: output_dir : ./output/ +[2022/04/18 10:11:26] root INFO: pretrained_model : None +[2022/04/18 10:11:26] root INFO: print_batch_step : 10 +[2022/04/18 10:11:26] root INFO: save_inference_dir : ./inference +[2022/04/18 10:11:26] root INFO: save_interval : 1 +[2022/04/18 10:11:26] root INFO: use_visualdl : False +[2022/04/18 10:11:26] root INFO: Infer : +[2022/04/18 10:11:26] root INFO: PostProcess : +[2022/04/18 10:11:26] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:11:26] root INFO: name : Topk +[2022/04/18 10:11:26] root INFO: topk : 5 +[2022/04/18 10:11:26] root INFO: batch_size : 10 +[2022/04/18 10:11:26] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:11:26] root INFO: transforms : +[2022/04/18 10:11:26] root INFO: DecodeImage : +[2022/04/18 10:11:26] root INFO: channel_first : False +[2022/04/18 10:11:26] root INFO: to_rgb : True +[2022/04/18 10:11:26] root INFO: ResizeImage : +[2022/04/18 10:11:26] root INFO: resize_short : 256 +[2022/04/18 10:11:26] root INFO: CropImage : +[2022/04/18 10:11:26] root INFO: size : 224 +[2022/04/18 10:11:26] root INFO: NormalizeImage : +[2022/04/18 10:11:26] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:11:26] root INFO: order : +[2022/04/18 10:11:26] root INFO: scale : 1.0/255.0 +[2022/04/18 10:11:26] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:11:26] root INFO: ToCHWImage : None +[2022/04/18 10:11:26] root INFO: Loss : +[2022/04/18 10:11:26] root INFO: Eval : +[2022/04/18 10:11:26] root INFO: CELoss : +[2022/04/18 10:11:26] root INFO: weight : 1.0 +[2022/04/18 10:11:26] root INFO: Train : +[2022/04/18 10:11:26] root INFO: CELoss : +[2022/04/18 10:11:26] root INFO: weight : 1.0 +[2022/04/18 10:11:26] root INFO: Metric : +[2022/04/18 10:11:26] root INFO: Eval : +[2022/04/18 10:11:26] root INFO: TopkAcc : +[2022/04/18 10:11:26] root INFO: topk : [1, 5] +[2022/04/18 10:11:26] root INFO: Train : +[2022/04/18 10:11:26] root INFO: TopkAcc : +[2022/04/18 10:11:26] root INFO: topk : [1, 5] +[2022/04/18 10:11:26] root INFO: Optimizer : +[2022/04/18 10:11:26] root INFO: lr : +[2022/04/18 10:11:26] root INFO: learning_rate : 0.0125 +[2022/04/18 10:11:26] root INFO: name : Cosine +[2022/04/18 10:11:26] root INFO: warmup_epoch : 5 +[2022/04/18 10:11:26] root INFO: momentum : 0.9 +[2022/04/18 10:11:26] root INFO: name : Momentum +[2022/04/18 10:11:26] root INFO: regularizer : +[2022/04/18 10:11:26] root INFO: coeff : 1e-05 +[2022/04/18 10:11:26] root INFO: name : L2 +[2022/04/18 10:11:26] root INFO: profiler_options : None +[2022/04/18 14:59:29] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 14:59:29] root INFO: Arch : +[2022/04/18 14:59:29] root INFO: class_num : 100 +[2022/04/18 14:59:29] root INFO: name : ResNet50_vd +[2022/04/18 14:59:29] root INFO: DataLoader : +[2022/04/18 14:59:29] root INFO: Eval : +[2022/04/18 14:59:29] root INFO: dataset : +[2022/04/18 14:59:29] root INFO: cls_label_path : ./dataset/CIFAR100/test_list.txt +[2022/04/18 14:59:29] root INFO: image_root : ./dataset/CIFAR100/ +[2022/04/18 14:59:29] root INFO: name : ImageNetDataset +[2022/04/18 14:59:29] root INFO: transform_ops : +[2022/04/18 14:59:29] root INFO: DecodeImage : +[2022/04/18 14:59:29] root INFO: channel_first : False +[2022/04/18 14:59:29] root INFO: to_rgb : True +[2022/04/18 14:59:29] root INFO: ResizeImage : +[2022/04/18 14:59:29] root INFO: resize_short : 36 +[2022/04/18 14:59:29] root INFO: CropImage : +[2022/04/18 14:59:29] root INFO: size : 32 +[2022/04/18 14:59:29] root INFO: NormalizeImage : +[2022/04/18 14:59:29] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 14:59:29] root INFO: order : +[2022/04/18 14:59:29] root INFO: scale : 1.0/255.0 +[2022/04/18 14:59:29] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 14:59:29] root INFO: loader : +[2022/04/18 14:59:29] root INFO: num_workers : 4 +[2022/04/18 14:59:29] root INFO: use_shared_memory : True +[2022/04/18 14:59:29] root INFO: sampler : +[2022/04/18 14:59:29] root INFO: batch_size : 64 +[2022/04/18 14:59:29] root INFO: drop_last : False +[2022/04/18 14:59:29] root INFO: name : DistributedBatchSampler +[2022/04/18 14:59:29] root INFO: shuffle : False +[2022/04/18 14:59:29] root INFO: Train : +[2022/04/18 14:59:29] root INFO: dataset : +[2022/04/18 14:59:29] root INFO: cls_label_path : ./dataset/CIFAR100/train_list.txt +[2022/04/18 14:59:29] root INFO: image_root : ./dataset/CIFAR100/ +[2022/04/18 14:59:29] root INFO: name : ImageNetDataset +[2022/04/18 14:59:29] root INFO: transform_ops : +[2022/04/18 14:59:29] root INFO: DecodeImage : +[2022/04/18 14:59:29] root INFO: channel_first : False +[2022/04/18 14:59:29] root INFO: to_rgb : True +[2022/04/18 14:59:29] root INFO: RandCropImage : +[2022/04/18 14:59:29] root INFO: size : 32 +[2022/04/18 14:59:29] root INFO: RandFlipImage : +[2022/04/18 14:59:29] root INFO: flip_code : 1 +[2022/04/18 14:59:29] root INFO: NormalizeImage : +[2022/04/18 14:59:29] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 14:59:29] root INFO: order : +[2022/04/18 14:59:29] root INFO: scale : 1.0/255.0 +[2022/04/18 14:59:29] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 14:59:29] root INFO: loader : +[2022/04/18 14:59:29] root INFO: num_workers : 4 +[2022/04/18 14:59:29] root INFO: use_shared_memory : True +[2022/04/18 14:59:29] root INFO: sampler : +[2022/04/18 14:59:29] root INFO: batch_size : 64 +[2022/04/18 14:59:29] root INFO: drop_last : False +[2022/04/18 14:59:29] root INFO: name : DistributedBatchSampler +[2022/04/18 14:59:29] root INFO: shuffle : True +[2022/04/18 14:59:29] root INFO: Global : +[2022/04/18 14:59:29] root INFO: checkpoints : None +[2022/04/18 14:59:29] root INFO: device : cpu +[2022/04/18 14:59:29] root INFO: epochs : 100 +[2022/04/18 14:59:29] root INFO: eval_during_train : True +[2022/04/18 14:59:29] root INFO: eval_interval : 1 +[2022/04/18 14:59:29] root INFO: image_shape : [3, 32, 32] +[2022/04/18 14:59:29] root INFO: output_dir : ./output/ +[2022/04/18 14:59:29] root INFO: pretrained_model : None +[2022/04/18 14:59:29] root INFO: print_batch_step : 10 +[2022/04/18 14:59:29] root INFO: save_inference_dir : ./inference +[2022/04/18 14:59:29] root INFO: save_interval : 1 +[2022/04/18 14:59:29] root INFO: use_visualdl : False +[2022/04/18 14:59:29] root INFO: Infer : +[2022/04/18 14:59:29] root INFO: PostProcess : +[2022/04/18 14:59:29] root INFO: name : Topk +[2022/04/18 14:59:29] root INFO: topk : 5 +[2022/04/18 14:59:29] root INFO: batch_size : 10 +[2022/04/18 14:59:29] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 14:59:29] root INFO: transforms : +[2022/04/18 14:59:29] root INFO: DecodeImage : +[2022/04/18 14:59:29] root INFO: channel_first : False +[2022/04/18 14:59:29] root INFO: to_rgb : True +[2022/04/18 14:59:29] root INFO: ResizeImage : +[2022/04/18 14:59:29] root INFO: resize_short : 36 +[2022/04/18 14:59:29] root INFO: CropImage : +[2022/04/18 14:59:29] root INFO: size : 32 +[2022/04/18 14:59:29] root INFO: NormalizeImage : +[2022/04/18 14:59:29] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 14:59:29] root INFO: order : +[2022/04/18 14:59:29] root INFO: scale : 1.0/255.0 +[2022/04/18 14:59:29] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 14:59:29] root INFO: ToCHWImage : None +[2022/04/18 14:59:29] root INFO: Loss : +[2022/04/18 14:59:29] root INFO: Eval : +[2022/04/18 14:59:29] root INFO: CELoss : +[2022/04/18 14:59:29] root INFO: weight : 1.0 +[2022/04/18 14:59:29] root INFO: Train : +[2022/04/18 14:59:29] root INFO: CELoss : +[2022/04/18 14:59:29] root INFO: weight : 1.0 +[2022/04/18 14:59:29] root INFO: Metric : +[2022/04/18 14:59:29] root INFO: Eval : +[2022/04/18 14:59:29] root INFO: TopkAcc : +[2022/04/18 14:59:29] root INFO: topk : [1, 5] +[2022/04/18 14:59:29] root INFO: Train : +[2022/04/18 14:59:29] root INFO: TopkAcc : +[2022/04/18 14:59:29] root INFO: topk : [1, 5] +[2022/04/18 14:59:29] root INFO: Optimizer : +[2022/04/18 14:59:29] root INFO: lr : +[2022/04/18 14:59:29] root INFO: learning_rate : 0.04 +[2022/04/18 14:59:29] root INFO: name : Cosine +[2022/04/18 14:59:29] root INFO: momentum : 0.9 +[2022/04/18 14:59:29] root INFO: name : Momentum +[2022/04/18 14:59:29] root INFO: regularizer : +[2022/04/18 14:59:29] root INFO: coeff : 0.0001 +[2022/04/18 14:59:29] root INFO: name : L2 +[2022/04/18 14:59:29] root INFO: output_dir : output_CIFAR +[2022/04/18 14:59:29] root INFO: profiler_options : None +[2022/04/18 14:59:29] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 14:59:42] root WARNING: The training strategy in config files provided by PaddleClas is based on 4 gpus. But the number of gpus is 1 in current training. Please modify the stategy (learning rate, batch size and so on) if use config files in PaddleClas to train. +[2022/04/18 14:59:45] root INFO: [Train][Epoch 1/100][Iter: 0/782]lr: 0.04000, top1: 0.00000, top5: 0.00000, CELoss: 5.06183, loss: 5.06183, batch_cost: 3.24200s, reader_cost: 0.06900, ips: 19.74091 images/sec, eta: 2 days, 22:25:24 +[2022/04/18 15:00:17] root INFO: [Train][Epoch 1/100][Iter: 10/782]lr: 0.04000, top1: 0.00568, top5: 0.04261, CELoss: 8.25212, loss: 8.25212, batch_cost: 3.30698s, reader_cost: 0.00026, ips: 19.35302 images/sec, eta: 2 days, 23:49:32 +[2022/04/18 15:00:50] root INFO: [Train][Epoch 1/100][Iter: 20/782]lr: 0.04000, top1: 0.00818, top5: 0.04539, CELoss: 8.45065, loss: 8.45065, batch_cost: 3.28671s, reader_cost: 0.00010, ips: 19.47239 images/sec, eta: 2 days, 23:22:34 diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdopt new file mode 100644 index 0000000..09e27e8 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdparams new file mode 100644 index 0000000..6758e33 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdstates new file mode 100644 index 0000000..e9570e6 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/best_model.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdopt new file mode 100644 index 0000000..75cc7be Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdparams new file mode 100644 index 0000000..6e1f30e Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdstates new file mode 100644 index 0000000..8fcacdd Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_1.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdopt new file mode 100644 index 0000000..0178570 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdparams new file mode 100644 index 0000000..ec2ac01 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdstates new file mode 100644 index 0000000..6f79548 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_10.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdopt new file mode 100644 index 0000000..09e27e8 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdparams new file mode 100644 index 0000000..6758e33 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdstates new file mode 100644 index 0000000..e9570e6 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_11.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdopt new file mode 100644 index 0000000..730fe89 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdparams new file mode 100644 index 0000000..aff0ae9 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdstates new file mode 100644 index 0000000..2d8fd78 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_12.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdopt new file mode 100644 index 0000000..19afdbb Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdparams new file mode 100644 index 0000000..d09dbce Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdstates new file mode 100644 index 0000000..82f77c9 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_13.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdopt new file mode 100644 index 0000000..d60f005 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdparams new file mode 100644 index 0000000..3522c94 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdstates new file mode 100644 index 0000000..063cc5b Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_14.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdopt new file mode 100644 index 0000000..c28e9a8 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdparams new file mode 100644 index 0000000..a69d18e Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdstates new file mode 100644 index 0000000..fb8d131 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_15.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdopt new file mode 100644 index 0000000..56cf0e5 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdparams new file mode 100644 index 0000000..87c4bbd Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdstates new file mode 100644 index 0000000..2140441 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_16.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdopt new file mode 100644 index 0000000..79523c6 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdparams new file mode 100644 index 0000000..095cdba Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdstates new file mode 100644 index 0000000..b9957e7 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_17.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdopt new file mode 100644 index 0000000..3eca271 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdparams new file mode 100644 index 0000000..6112fd8 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdstates new file mode 100644 index 0000000..5613ff6 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_18.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdopt new file mode 100644 index 0000000..e6e9d71 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdparams new file mode 100644 index 0000000..e2c1fe6 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdstates new file mode 100644 index 0000000..a4cf257 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_19.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdopt new file mode 100644 index 0000000..84d7c65 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdparams new file mode 100644 index 0000000..1cb95fe Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdstates new file mode 100644 index 0000000..fcd4c7d Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_2.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdopt new file mode 100644 index 0000000..34723e3 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdparams new file mode 100644 index 0000000..1642e22 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdstates new file mode 100644 index 0000000..80a0d80 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_20.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdopt new file mode 100644 index 0000000..3e465c9 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdparams new file mode 100644 index 0000000..0e67441 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdstates new file mode 100644 index 0000000..33f8acd Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_3.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdopt new file mode 100644 index 0000000..89b4c36 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdparams new file mode 100644 index 0000000..b4c4e62 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdstates new file mode 100644 index 0000000..e7b9810 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_4.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdopt new file mode 100644 index 0000000..5bbdaf5 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdparams new file mode 100644 index 0000000..93e2814 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdstates new file mode 100644 index 0000000..e128df0 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_5.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdopt new file mode 100644 index 0000000..d6380fa Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdparams new file mode 100644 index 0000000..36ce58f Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdstates new file mode 100644 index 0000000..70cd892 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_6.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdopt new file mode 100644 index 0000000..943c25e Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdparams new file mode 100644 index 0000000..d763f08 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdstates new file mode 100644 index 0000000..782379c Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_7.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdopt new file mode 100644 index 0000000..03395f7 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdparams new file mode 100644 index 0000000..fd31025 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdstates new file mode 100644 index 0000000..4eb02df Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_8.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdopt new file mode 100644 index 0000000..345829b Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdparams new file mode 100644 index 0000000..5a75a93 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdstates new file mode 100644 index 0000000..7fcee30 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/epoch_9.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/infer.log b/src/PaddleClas/output/ShuffleNetV2_x0_25/infer.log new file mode 100644 index 0000000..5a92c42 --- /dev/null +++ b/src/PaddleClas/output/ShuffleNetV2_x0_25/infer.log @@ -0,0 +1,125 @@ +[2022/04/18 10:43:55] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:43:55] root INFO: Arch : +[2022/04/18 10:43:55] root INFO: class_num : 102 +[2022/04/18 10:43:55] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:43:55] root INFO: DataLoader : +[2022/04/18 10:43:55] root INFO: Eval : +[2022/04/18 10:43:55] root INFO: dataset : +[2022/04/18 10:43:55] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:43:55] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:43:55] root INFO: name : ImageNetDataset +[2022/04/18 10:43:55] root INFO: transform_ops : +[2022/04/18 10:43:55] root INFO: DecodeImage : +[2022/04/18 10:43:55] root INFO: channel_first : False +[2022/04/18 10:43:55] root INFO: to_rgb : True +[2022/04/18 10:43:55] root INFO: ResizeImage : +[2022/04/18 10:43:55] root INFO: resize_short : 256 +[2022/04/18 10:43:55] root INFO: CropImage : +[2022/04/18 10:43:55] root INFO: size : 224 +[2022/04/18 10:43:55] root INFO: NormalizeImage : +[2022/04/18 10:43:55] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:43:55] root INFO: order : +[2022/04/18 10:43:55] root INFO: scale : 1.0/255.0 +[2022/04/18 10:43:55] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:43:55] root INFO: loader : +[2022/04/18 10:43:55] root INFO: num_workers : 4 +[2022/04/18 10:43:55] root INFO: use_shared_memory : True +[2022/04/18 10:43:55] root INFO: sampler : +[2022/04/18 10:43:55] root INFO: batch_size : 64 +[2022/04/18 10:43:55] root INFO: drop_last : False +[2022/04/18 10:43:55] root INFO: name : DistributedBatchSampler +[2022/04/18 10:43:55] root INFO: shuffle : False +[2022/04/18 10:43:55] root INFO: Train : +[2022/04/18 10:43:55] root INFO: dataset : +[2022/04/18 10:43:55] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:43:55] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:43:55] root INFO: name : ImageNetDataset +[2022/04/18 10:43:55] root INFO: transform_ops : +[2022/04/18 10:43:55] root INFO: DecodeImage : +[2022/04/18 10:43:55] root INFO: channel_first : False +[2022/04/18 10:43:55] root INFO: to_rgb : True +[2022/04/18 10:43:55] root INFO: RandCropImage : +[2022/04/18 10:43:55] root INFO: size : 224 +[2022/04/18 10:43:55] root INFO: RandFlipImage : +[2022/04/18 10:43:55] root INFO: flip_code : 1 +[2022/04/18 10:43:55] root INFO: NormalizeImage : +[2022/04/18 10:43:55] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:43:55] root INFO: order : +[2022/04/18 10:43:55] root INFO: scale : 1.0/255.0 +[2022/04/18 10:43:55] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:43:55] root INFO: loader : +[2022/04/18 10:43:55] root INFO: num_workers : 4 +[2022/04/18 10:43:55] root INFO: use_shared_memory : True +[2022/04/18 10:43:55] root INFO: sampler : +[2022/04/18 10:43:55] root INFO: batch_size : 256 +[2022/04/18 10:43:55] root INFO: drop_last : False +[2022/04/18 10:43:55] root INFO: name : DistributedBatchSampler +[2022/04/18 10:43:55] root INFO: shuffle : True +[2022/04/18 10:43:55] root INFO: Global : +[2022/04/18 10:43:55] root INFO: checkpoints : None +[2022/04/18 10:43:55] root INFO: device : cpu +[2022/04/18 10:43:55] root INFO: epochs : 20 +[2022/04/18 10:43:55] root INFO: eval_during_train : True +[2022/04/18 10:43:55] root INFO: eval_interval : 1 +[2022/04/18 10:43:55] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:43:55] root INFO: output_dir : ./output/ +[2022/04/18 10:43:55] root INFO: pretrained_model : output/ShuffleNetV2_x0_25/best_model +[2022/04/18 10:43:55] root INFO: print_batch_step : 10 +[2022/04/18 10:43:55] root INFO: save_inference_dir : ./inference +[2022/04/18 10:43:55] root INFO: save_interval : 1 +[2022/04/18 10:43:55] root INFO: use_visualdl : False +[2022/04/18 10:43:55] root INFO: Infer : +[2022/04/18 10:43:55] root INFO: PostProcess : +[2022/04/18 10:43:55] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:43:55] root INFO: name : Topk +[2022/04/18 10:43:55] root INFO: topk : 5 +[2022/04/18 10:43:55] root INFO: batch_size : 10 +[2022/04/18 10:43:55] root INFO: infer_imgs : dataset/flowers102/jpg/image_00001.jpg +[2022/04/18 10:43:55] root INFO: transforms : +[2022/04/18 10:43:55] root INFO: DecodeImage : +[2022/04/18 10:43:55] root INFO: channel_first : False +[2022/04/18 10:43:55] root INFO: to_rgb : True +[2022/04/18 10:43:55] root INFO: ResizeImage : +[2022/04/18 10:43:55] root INFO: resize_short : 256 +[2022/04/18 10:43:55] root INFO: CropImage : +[2022/04/18 10:43:55] root INFO: size : 224 +[2022/04/18 10:43:55] root INFO: NormalizeImage : +[2022/04/18 10:43:55] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:43:55] root INFO: order : +[2022/04/18 10:43:55] root INFO: scale : 1.0/255.0 +[2022/04/18 10:43:55] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:43:55] root INFO: ToCHWImage : None +[2022/04/18 10:43:55] root INFO: Loss : +[2022/04/18 10:43:55] root INFO: Eval : +[2022/04/18 10:43:55] root INFO: CELoss : +[2022/04/18 10:43:55] root INFO: weight : 1.0 +[2022/04/18 10:43:55] root INFO: Train : +[2022/04/18 10:43:55] root INFO: CELoss : +[2022/04/18 10:43:55] root INFO: weight : 1.0 +[2022/04/18 10:43:55] root INFO: Metric : +[2022/04/18 10:43:55] root INFO: Eval : +[2022/04/18 10:43:55] root INFO: TopkAcc : +[2022/04/18 10:43:55] root INFO: topk : [1, 5] +[2022/04/18 10:43:55] root INFO: Train : +[2022/04/18 10:43:55] root INFO: TopkAcc : +[2022/04/18 10:43:55] root INFO: topk : [1, 5] +[2022/04/18 10:43:55] root INFO: Optimizer : +[2022/04/18 10:43:55] root INFO: lr : +[2022/04/18 10:43:55] root INFO: learning_rate : 0.0125 +[2022/04/18 10:43:55] root INFO: name : Cosine +[2022/04/18 10:43:55] root INFO: warmup_epoch : 5 +[2022/04/18 10:43:55] root INFO: momentum : 0.9 +[2022/04/18 10:43:55] root INFO: name : Momentum +[2022/04/18 10:43:55] root INFO: regularizer : +[2022/04/18 10:43:55] root INFO: coeff : 1e-05 +[2022/04/18 10:43:55] root INFO: name : L2 +[2022/04/18 10:43:55] root INFO: train with paddle 2.2.2 and device CPUPlace diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdopt b/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdopt new file mode 100644 index 0000000..34723e3 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdopt differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdparams b/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdparams new file mode 100644 index 0000000..1642e22 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdparams differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdstates b/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdstates new file mode 100644 index 0000000..80a0d80 Binary files /dev/null and b/src/PaddleClas/output/ShuffleNetV2_x0_25/latest.pdstates differ diff --git a/src/PaddleClas/output/ShuffleNetV2_x0_25/train.log b/src/PaddleClas/output/ShuffleNetV2_x0_25/train.log new file mode 100644 index 0000000..769385f --- /dev/null +++ b/src/PaddleClas/output/ShuffleNetV2_x0_25/train.log @@ -0,0 +1,1427 @@ +[2022/04/18 10:08:38] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:08:38] root INFO: Arch : +[2022/04/18 10:08:38] root INFO: class_num : 102 +[2022/04/18 10:08:38] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:08:38] root INFO: DataLoader : +[2022/04/18 10:08:38] root INFO: Eval : +[2022/04/18 10:08:38] root INFO: dataset : +[2022/04/18 10:08:38] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:08:38] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:08:38] root INFO: name : ImageNetDataset +[2022/04/18 10:08:38] root INFO: transform_ops : +[2022/04/18 10:08:38] root INFO: DecodeImage : +[2022/04/18 10:08:38] root INFO: channel_first : False +[2022/04/18 10:08:38] root INFO: to_rgb : True +[2022/04/18 10:08:38] root INFO: ResizeImage : +[2022/04/18 10:08:38] root INFO: resize_short : 256 +[2022/04/18 10:08:38] root INFO: CropImage : +[2022/04/18 10:08:38] root INFO: size : 224 +[2022/04/18 10:08:38] root INFO: NormalizeImage : +[2022/04/18 10:08:38] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:08:38] root INFO: order : +[2022/04/18 10:08:38] root INFO: scale : 1.0/255.0 +[2022/04/18 10:08:38] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:08:38] root INFO: loader : +[2022/04/18 10:08:38] root INFO: num_workers : 4 +[2022/04/18 10:08:38] root INFO: use_shared_memory : True +[2022/04/18 10:08:38] root INFO: sampler : +[2022/04/18 10:08:38] root INFO: batch_size : 64 +[2022/04/18 10:08:38] root INFO: drop_last : False +[2022/04/18 10:08:38] root INFO: name : DistributedBatchSampler +[2022/04/18 10:08:38] root INFO: shuffle : False +[2022/04/18 10:08:38] root INFO: Train : +[2022/04/18 10:08:38] root INFO: dataset : +[2022/04/18 10:08:38] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:08:38] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:08:38] root INFO: name : ImageNetDataset +[2022/04/18 10:08:38] root INFO: transform_ops : +[2022/04/18 10:08:38] root INFO: DecodeImage : +[2022/04/18 10:08:38] root INFO: channel_first : False +[2022/04/18 10:08:38] root INFO: to_rgb : True +[2022/04/18 10:08:38] root INFO: RandCropImage : +[2022/04/18 10:08:38] root INFO: size : 224 +[2022/04/18 10:08:38] root INFO: RandFlipImage : +[2022/04/18 10:08:38] root INFO: flip_code : 1 +[2022/04/18 10:08:38] root INFO: NormalizeImage : +[2022/04/18 10:08:38] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:08:38] root INFO: order : +[2022/04/18 10:08:38] root INFO: scale : 1.0/255.0 +[2022/04/18 10:08:38] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:08:38] root INFO: loader : +[2022/04/18 10:08:38] root INFO: num_workers : 4 +[2022/04/18 10:08:38] root INFO: use_shared_memory : True +[2022/04/18 10:08:38] root INFO: sampler : +[2022/04/18 10:08:38] root INFO: batch_size : 256 +[2022/04/18 10:08:38] root INFO: drop_last : False +[2022/04/18 10:08:38] root INFO: name : DistributedBatchSampler +[2022/04/18 10:08:38] root INFO: shuffle : True +[2022/04/18 10:08:38] root INFO: Global : +[2022/04/18 10:08:38] root INFO: checkpoints : None +[2022/04/18 10:08:38] root INFO: device : cpu +[2022/04/18 10:08:38] root INFO: epochs : 20 +[2022/04/18 10:08:38] root INFO: eval_during_train : True +[2022/04/18 10:08:38] root INFO: eval_interval : 1 +[2022/04/18 10:08:38] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:08:38] root INFO: output_dir : ./output/ +[2022/04/18 10:08:38] root INFO: pretrained_model : None +[2022/04/18 10:08:38] root INFO: print_batch_step : 10 +[2022/04/18 10:08:38] root INFO: save_inference_dir : ./inference +[2022/04/18 10:08:38] root INFO: save_interval : 1 +[2022/04/18 10:08:38] root INFO: use_visualdl : False +[2022/04/18 10:08:38] root INFO: Infer : +[2022/04/18 10:08:38] root INFO: PostProcess : +[2022/04/18 10:08:38] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:08:38] root INFO: name : Topk +[2022/04/18 10:08:38] root INFO: topk : 5 +[2022/04/18 10:08:38] root INFO: batch_size : 10 +[2022/04/18 10:08:38] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:08:38] root INFO: transforms : +[2022/04/18 10:08:38] root INFO: DecodeImage : +[2022/04/18 10:08:38] root INFO: channel_first : False +[2022/04/18 10:08:38] root INFO: to_rgb : True +[2022/04/18 10:08:38] root INFO: ResizeImage : +[2022/04/18 10:08:38] root INFO: resize_short : 256 +[2022/04/18 10:08:38] root INFO: CropImage : +[2022/04/18 10:08:38] root INFO: size : 224 +[2022/04/18 10:08:38] root INFO: NormalizeImage : +[2022/04/18 10:08:38] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:08:38] root INFO: order : +[2022/04/18 10:08:38] root INFO: scale : 1.0/255.0 +[2022/04/18 10:08:38] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:08:38] root INFO: ToCHWImage : None +[2022/04/18 10:08:38] root INFO: Loss : +[2022/04/18 10:08:38] root INFO: Eval : +[2022/04/18 10:08:38] root INFO: CELoss : +[2022/04/18 10:08:38] root INFO: weight : 1.0 +[2022/04/18 10:08:38] root INFO: Train : +[2022/04/18 10:08:38] root INFO: CELoss : +[2022/04/18 10:08:38] root INFO: weight : 1.0 +[2022/04/18 10:08:38] root INFO: Metric : +[2022/04/18 10:08:38] root INFO: Eval : +[2022/04/18 10:08:38] root INFO: TopkAcc : +[2022/04/18 10:08:38] root INFO: topk : [1, 5] +[2022/04/18 10:08:38] root INFO: Train : +[2022/04/18 10:08:38] root INFO: TopkAcc : +[2022/04/18 10:08:38] root INFO: topk : [1, 5] +[2022/04/18 10:08:38] root INFO: Optimizer : +[2022/04/18 10:08:38] root INFO: lr : +[2022/04/18 10:08:38] root INFO: learning_rate : 0.0125 +[2022/04/18 10:08:38] root INFO: name : Cosine +[2022/04/18 10:08:38] root INFO: warmup_epoch : 5 +[2022/04/18 10:08:38] root INFO: momentum : 0.9 +[2022/04/18 10:08:38] root INFO: name : Momentum +[2022/04/18 10:08:38] root INFO: regularizer : +[2022/04/18 10:08:38] root INFO: coeff : 1e-05 +[2022/04/18 10:08:38] root INFO: name : L2 +[2022/04/18 10:08:38] root INFO: profiler_options : None +[2022/04/18 10:08:38] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:10:05] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:10:05] root INFO: Arch : +[2022/04/18 10:10:05] root INFO: class_num : 102 +[2022/04/18 10:10:05] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:10:05] root INFO: pretrained : True +[2022/04/18 10:10:05] root INFO: DataLoader : +[2022/04/18 10:10:05] root INFO: Eval : +[2022/04/18 10:10:05] root INFO: dataset : +[2022/04/18 10:10:05] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:10:05] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:10:05] root INFO: name : ImageNetDataset +[2022/04/18 10:10:05] root INFO: transform_ops : +[2022/04/18 10:10:05] root INFO: DecodeImage : +[2022/04/18 10:10:05] root INFO: channel_first : False +[2022/04/18 10:10:05] root INFO: to_rgb : True +[2022/04/18 10:10:05] root INFO: ResizeImage : +[2022/04/18 10:10:05] root INFO: resize_short : 256 +[2022/04/18 10:10:05] root INFO: CropImage : +[2022/04/18 10:10:05] root INFO: size : 224 +[2022/04/18 10:10:05] root INFO: NormalizeImage : +[2022/04/18 10:10:05] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:05] root INFO: order : +[2022/04/18 10:10:05] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:05] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:05] root INFO: loader : +[2022/04/18 10:10:05] root INFO: num_workers : 4 +[2022/04/18 10:10:05] root INFO: use_shared_memory : True +[2022/04/18 10:10:05] root INFO: sampler : +[2022/04/18 10:10:05] root INFO: batch_size : 64 +[2022/04/18 10:10:05] root INFO: drop_last : False +[2022/04/18 10:10:05] root INFO: name : DistributedBatchSampler +[2022/04/18 10:10:05] root INFO: shuffle : False +[2022/04/18 10:10:05] root INFO: Train : +[2022/04/18 10:10:05] root INFO: dataset : +[2022/04/18 10:10:05] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:10:05] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:10:05] root INFO: name : ImageNetDataset +[2022/04/18 10:10:05] root INFO: transform_ops : +[2022/04/18 10:10:05] root INFO: DecodeImage : +[2022/04/18 10:10:05] root INFO: channel_first : False +[2022/04/18 10:10:05] root INFO: to_rgb : True +[2022/04/18 10:10:05] root INFO: RandCropImage : +[2022/04/18 10:10:05] root INFO: size : 224 +[2022/04/18 10:10:05] root INFO: RandFlipImage : +[2022/04/18 10:10:05] root INFO: flip_code : 1 +[2022/04/18 10:10:05] root INFO: NormalizeImage : +[2022/04/18 10:10:05] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:05] root INFO: order : +[2022/04/18 10:10:05] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:05] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:05] root INFO: loader : +[2022/04/18 10:10:05] root INFO: num_workers : 4 +[2022/04/18 10:10:05] root INFO: use_shared_memory : True +[2022/04/18 10:10:05] root INFO: sampler : +[2022/04/18 10:10:05] root INFO: batch_size : 256 +[2022/04/18 10:10:05] root INFO: drop_last : False +[2022/04/18 10:10:05] root INFO: name : DistributedBatchSampler +[2022/04/18 10:10:05] root INFO: shuffle : True +[2022/04/18 10:10:05] root INFO: Global : +[2022/04/18 10:10:05] root INFO: checkpoints : None +[2022/04/18 10:10:05] root INFO: device : cpu +[2022/04/18 10:10:05] root INFO: epochs : 20 +[2022/04/18 10:10:05] root INFO: eval_during_train : True +[2022/04/18 10:10:05] root INFO: eval_interval : 1 +[2022/04/18 10:10:05] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:10:05] root INFO: output_dir : ./output/ +[2022/04/18 10:10:05] root INFO: pretrained_model : None +[2022/04/18 10:10:05] root INFO: print_batch_step : 10 +[2022/04/18 10:10:05] root INFO: save_inference_dir : ./inference +[2022/04/18 10:10:05] root INFO: save_interval : 1 +[2022/04/18 10:10:05] root INFO: use_visualdl : False +[2022/04/18 10:10:05] root INFO: Infer : +[2022/04/18 10:10:05] root INFO: PostProcess : +[2022/04/18 10:10:05] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:10:05] root INFO: name : Topk +[2022/04/18 10:10:05] root INFO: topk : 5 +[2022/04/18 10:10:05] root INFO: batch_size : 10 +[2022/04/18 10:10:05] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:10:05] root INFO: transforms : +[2022/04/18 10:10:05] root INFO: DecodeImage : +[2022/04/18 10:10:05] root INFO: channel_first : False +[2022/04/18 10:10:05] root INFO: to_rgb : True +[2022/04/18 10:10:05] root INFO: ResizeImage : +[2022/04/18 10:10:05] root INFO: resize_short : 256 +[2022/04/18 10:10:05] root INFO: CropImage : +[2022/04/18 10:10:05] root INFO: size : 224 +[2022/04/18 10:10:05] root INFO: NormalizeImage : +[2022/04/18 10:10:05] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:05] root INFO: order : +[2022/04/18 10:10:05] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:05] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:05] root INFO: ToCHWImage : None +[2022/04/18 10:10:05] root INFO: Loss : +[2022/04/18 10:10:05] root INFO: Eval : +[2022/04/18 10:10:05] root INFO: CELoss : +[2022/04/18 10:10:05] root INFO: weight : 1.0 +[2022/04/18 10:10:05] root INFO: Train : +[2022/04/18 10:10:05] root INFO: CELoss : +[2022/04/18 10:10:05] root INFO: weight : 1.0 +[2022/04/18 10:10:05] root INFO: Metric : +[2022/04/18 10:10:05] root INFO: Eval : +[2022/04/18 10:10:05] root INFO: TopkAcc : +[2022/04/18 10:10:05] root INFO: topk : [1, 5] +[2022/04/18 10:10:05] root INFO: Train : +[2022/04/18 10:10:05] root INFO: TopkAcc : +[2022/04/18 10:10:05] root INFO: topk : [1, 5] +[2022/04/18 10:10:05] root INFO: Optimizer : +[2022/04/18 10:10:05] root INFO: lr : +[2022/04/18 10:10:05] root INFO: learning_rate : 0.0125 +[2022/04/18 10:10:05] root INFO: name : Cosine +[2022/04/18 10:10:05] root INFO: warmup_epoch : 5 +[2022/04/18 10:10:05] root INFO: momentum : 0.9 +[2022/04/18 10:10:05] root INFO: name : Momentum +[2022/04/18 10:10:05] root INFO: regularizer : +[2022/04/18 10:10:05] root INFO: coeff : 1e-05 +[2022/04/18 10:10:05] root INFO: name : L2 +[2022/04/18 10:10:05] root INFO: profiler_options : None +[2022/04/18 10:10:05] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:10:31] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:10:31] root INFO: Arch : +[2022/04/18 10:10:31] root INFO: class_num : 102 +[2022/04/18 10:10:31] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:10:31] root INFO: pretrained : True +[2022/04/18 10:10:31] root INFO: DataLoader : +[2022/04/18 10:10:31] root INFO: Eval : +[2022/04/18 10:10:31] root INFO: dataset : +[2022/04/18 10:10:31] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:10:31] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:10:31] root INFO: name : ImageNetDataset +[2022/04/18 10:10:31] root INFO: transform_ops : +[2022/04/18 10:10:31] root INFO: DecodeImage : +[2022/04/18 10:10:31] root INFO: channel_first : False +[2022/04/18 10:10:31] root INFO: to_rgb : True +[2022/04/18 10:10:31] root INFO: ResizeImage : +[2022/04/18 10:10:31] root INFO: resize_short : 256 +[2022/04/18 10:10:31] root INFO: CropImage : +[2022/04/18 10:10:31] root INFO: size : 224 +[2022/04/18 10:10:31] root INFO: NormalizeImage : +[2022/04/18 10:10:31] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:31] root INFO: order : +[2022/04/18 10:10:31] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:31] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:31] root INFO: loader : +[2022/04/18 10:10:31] root INFO: num_workers : 4 +[2022/04/18 10:10:31] root INFO: use_shared_memory : True +[2022/04/18 10:10:31] root INFO: sampler : +[2022/04/18 10:10:31] root INFO: batch_size : 64 +[2022/04/18 10:10:31] root INFO: drop_last : False +[2022/04/18 10:10:31] root INFO: name : DistributedBatchSampler +[2022/04/18 10:10:31] root INFO: shuffle : False +[2022/04/18 10:10:31] root INFO: Train : +[2022/04/18 10:10:31] root INFO: dataset : +[2022/04/18 10:10:31] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:10:31] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:10:31] root INFO: name : ImageNetDataset +[2022/04/18 10:10:31] root INFO: transform_ops : +[2022/04/18 10:10:31] root INFO: DecodeImage : +[2022/04/18 10:10:31] root INFO: channel_first : False +[2022/04/18 10:10:31] root INFO: to_rgb : True +[2022/04/18 10:10:31] root INFO: RandCropImage : +[2022/04/18 10:10:31] root INFO: size : 224 +[2022/04/18 10:10:31] root INFO: RandFlipImage : +[2022/04/18 10:10:31] root INFO: flip_code : 1 +[2022/04/18 10:10:31] root INFO: NormalizeImage : +[2022/04/18 10:10:31] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:31] root INFO: order : +[2022/04/18 10:10:31] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:31] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:31] root INFO: loader : +[2022/04/18 10:10:31] root INFO: num_workers : 4 +[2022/04/18 10:10:31] root INFO: use_shared_memory : True +[2022/04/18 10:10:31] root INFO: sampler : +[2022/04/18 10:10:31] root INFO: batch_size : 256 +[2022/04/18 10:10:31] root INFO: drop_last : False +[2022/04/18 10:10:31] root INFO: name : DistributedBatchSampler +[2022/04/18 10:10:31] root INFO: shuffle : True +[2022/04/18 10:10:31] root INFO: Global : +[2022/04/18 10:10:31] root INFO: checkpoints : None +[2022/04/18 10:10:31] root INFO: device : cpu +[2022/04/18 10:10:31] root INFO: epochs : 20 +[2022/04/18 10:10:31] root INFO: eval_during_train : True +[2022/04/18 10:10:31] root INFO: eval_interval : 1 +[2022/04/18 10:10:31] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:10:31] root INFO: output_dir : ./output/ +[2022/04/18 10:10:31] root INFO: pretrained_model : None +[2022/04/18 10:10:31] root INFO: print_batch_step : 10 +[2022/04/18 10:10:31] root INFO: save_inference_dir : ./inference +[2022/04/18 10:10:31] root INFO: save_interval : 1 +[2022/04/18 10:10:31] root INFO: use_visualdl : False +[2022/04/18 10:10:31] root INFO: Infer : +[2022/04/18 10:10:31] root INFO: PostProcess : +[2022/04/18 10:10:31] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:10:31] root INFO: name : Topk +[2022/04/18 10:10:31] root INFO: topk : 5 +[2022/04/18 10:10:31] root INFO: batch_size : 10 +[2022/04/18 10:10:31] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:10:31] root INFO: transforms : +[2022/04/18 10:10:31] root INFO: DecodeImage : +[2022/04/18 10:10:31] root INFO: channel_first : False +[2022/04/18 10:10:31] root INFO: to_rgb : True +[2022/04/18 10:10:31] root INFO: ResizeImage : +[2022/04/18 10:10:31] root INFO: resize_short : 256 +[2022/04/18 10:10:31] root INFO: CropImage : +[2022/04/18 10:10:31] root INFO: size : 224 +[2022/04/18 10:10:31] root INFO: NormalizeImage : +[2022/04/18 10:10:31] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:10:31] root INFO: order : +[2022/04/18 10:10:31] root INFO: scale : 1.0/255.0 +[2022/04/18 10:10:31] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:10:31] root INFO: ToCHWImage : None +[2022/04/18 10:10:31] root INFO: Loss : +[2022/04/18 10:10:31] root INFO: Eval : +[2022/04/18 10:10:31] root INFO: CELoss : +[2022/04/18 10:10:31] root INFO: weight : 1.0 +[2022/04/18 10:10:31] root INFO: Train : +[2022/04/18 10:10:31] root INFO: CELoss : +[2022/04/18 10:10:31] root INFO: weight : 1.0 +[2022/04/18 10:10:31] root INFO: Metric : +[2022/04/18 10:10:31] root INFO: Eval : +[2022/04/18 10:10:31] root INFO: TopkAcc : +[2022/04/18 10:10:31] root INFO: topk : [1, 5] +[2022/04/18 10:10:31] root INFO: Train : +[2022/04/18 10:10:31] root INFO: TopkAcc : +[2022/04/18 10:10:31] root INFO: topk : [1, 5] +[2022/04/18 10:10:31] root INFO: Optimizer : +[2022/04/18 10:10:31] root INFO: lr : +[2022/04/18 10:10:31] root INFO: learning_rate : 0.0125 +[2022/04/18 10:10:31] root INFO: name : Cosine +[2022/04/18 10:10:31] root INFO: warmup_epoch : 5 +[2022/04/18 10:10:31] root INFO: momentum : 0.9 +[2022/04/18 10:10:31] root INFO: name : Momentum +[2022/04/18 10:10:31] root INFO: regularizer : +[2022/04/18 10:10:31] root INFO: coeff : 1e-05 +[2022/04/18 10:10:31] root INFO: name : L2 +[2022/04/18 10:10:31] root INFO: profiler_options : None +[2022/04/18 10:10:31] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:17:07] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:17:07] root INFO: Arch : +[2022/04/18 10:17:07] root INFO: class_num : 102 +[2022/04/18 10:17:07] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:17:07] root INFO: DataLoader : +[2022/04/18 10:17:07] root INFO: Eval : +[2022/04/18 10:17:07] root INFO: dataset : +[2022/04/18 10:17:07] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:17:07] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:17:07] root INFO: name : ImageNetDataset +[2022/04/18 10:17:07] root INFO: transform_ops : +[2022/04/18 10:17:07] root INFO: DecodeImage : +[2022/04/18 10:17:07] root INFO: channel_first : False +[2022/04/18 10:17:07] root INFO: to_rgb : True +[2022/04/18 10:17:07] root INFO: ResizeImage : +[2022/04/18 10:17:07] root INFO: resize_short : 256 +[2022/04/18 10:17:07] root INFO: CropImage : +[2022/04/18 10:17:07] root INFO: size : 224 +[2022/04/18 10:17:07] root INFO: NormalizeImage : +[2022/04/18 10:17:07] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:17:07] root INFO: order : +[2022/04/18 10:17:07] root INFO: scale : 1.0/255.0 +[2022/04/18 10:17:07] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:17:07] root INFO: loader : +[2022/04/18 10:17:07] root INFO: num_workers : 4 +[2022/04/18 10:17:07] root INFO: use_shared_memory : True +[2022/04/18 10:17:07] root INFO: sampler : +[2022/04/18 10:17:07] root INFO: batch_size : 64 +[2022/04/18 10:17:07] root INFO: drop_last : False +[2022/04/18 10:17:07] root INFO: name : DistributedBatchSampler +[2022/04/18 10:17:07] root INFO: shuffle : False +[2022/04/18 10:17:07] root INFO: Train : +[2022/04/18 10:17:07] root INFO: dataset : +[2022/04/18 10:17:07] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:17:07] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:17:07] root INFO: name : ImageNetDataset +[2022/04/18 10:17:07] root INFO: transform_ops : +[2022/04/18 10:17:07] root INFO: DecodeImage : +[2022/04/18 10:17:07] root INFO: channel_first : False +[2022/04/18 10:17:07] root INFO: to_rgb : True +[2022/04/18 10:17:07] root INFO: RandCropImage : +[2022/04/18 10:17:07] root INFO: size : 224 +[2022/04/18 10:17:07] root INFO: RandFlipImage : +[2022/04/18 10:17:07] root INFO: flip_code : 1 +[2022/04/18 10:17:07] root INFO: NormalizeImage : +[2022/04/18 10:17:07] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:17:07] root INFO: order : +[2022/04/18 10:17:07] root INFO: scale : 1.0/255.0 +[2022/04/18 10:17:07] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:17:07] root INFO: loader : +[2022/04/18 10:17:07] root INFO: num_workers : 4 +[2022/04/18 10:17:07] root INFO: use_shared_memory : True +[2022/04/18 10:17:07] root INFO: sampler : +[2022/04/18 10:17:07] root INFO: batch_size : 256 +[2022/04/18 10:17:07] root INFO: drop_last : False +[2022/04/18 10:17:07] root INFO: name : DistributedBatchSampler +[2022/04/18 10:17:07] root INFO: shuffle : True +[2022/04/18 10:17:07] root INFO: Global : +[2022/04/18 10:17:07] root INFO: checkpoints : None +[2022/04/18 10:17:07] root INFO: device : cpu +[2022/04/18 10:17:07] root INFO: epochs : 20 +[2022/04/18 10:17:07] root INFO: eval_during_train : True +[2022/04/18 10:17:07] root INFO: eval_interval : 1 +[2022/04/18 10:17:07] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:17:07] root INFO: output_dir : ./output/ +[2022/04/18 10:17:07] root INFO: pretrained_model : None +[2022/04/18 10:17:07] root INFO: print_batch_step : 10 +[2022/04/18 10:17:07] root INFO: save_inference_dir : ./inference +[2022/04/18 10:17:07] root INFO: save_interval : 1 +[2022/04/18 10:17:07] root INFO: use_visualdl : False +[2022/04/18 10:17:07] root INFO: Infer : +[2022/04/18 10:17:07] root INFO: PostProcess : +[2022/04/18 10:17:07] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:17:07] root INFO: name : Topk +[2022/04/18 10:17:07] root INFO: topk : 5 +[2022/04/18 10:17:07] root INFO: batch_size : 10 +[2022/04/18 10:17:07] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:17:07] root INFO: transforms : +[2022/04/18 10:17:07] root INFO: DecodeImage : +[2022/04/18 10:17:07] root INFO: channel_first : False +[2022/04/18 10:17:07] root INFO: to_rgb : True +[2022/04/18 10:17:07] root INFO: ResizeImage : +[2022/04/18 10:17:07] root INFO: resize_short : 256 +[2022/04/18 10:17:07] root INFO: CropImage : +[2022/04/18 10:17:07] root INFO: size : 224 +[2022/04/18 10:17:07] root INFO: NormalizeImage : +[2022/04/18 10:17:07] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:17:07] root INFO: order : +[2022/04/18 10:17:07] root INFO: scale : 1.0/255.0 +[2022/04/18 10:17:07] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:17:07] root INFO: ToCHWImage : None +[2022/04/18 10:17:07] root INFO: Loss : +[2022/04/18 10:17:07] root INFO: Eval : +[2022/04/18 10:17:07] root INFO: CELoss : +[2022/04/18 10:17:07] root INFO: weight : 1.0 +[2022/04/18 10:17:07] root INFO: Train : +[2022/04/18 10:17:07] root INFO: CELoss : +[2022/04/18 10:17:07] root INFO: weight : 1.0 +[2022/04/18 10:17:07] root INFO: Metric : +[2022/04/18 10:17:07] root INFO: Eval : +[2022/04/18 10:17:07] root INFO: TopkAcc : +[2022/04/18 10:17:07] root INFO: topk : [1, 5] +[2022/04/18 10:17:07] root INFO: Train : +[2022/04/18 10:17:07] root INFO: TopkAcc : +[2022/04/18 10:17:07] root INFO: topk : [1, 5] +[2022/04/18 10:17:07] root INFO: Optimizer : +[2022/04/18 10:17:07] root INFO: lr : +[2022/04/18 10:17:07] root INFO: learning_rate : 0.0125 +[2022/04/18 10:17:07] root INFO: name : Cosine +[2022/04/18 10:17:07] root INFO: warmup_epoch : 5 +[2022/04/18 10:17:07] root INFO: momentum : 0.9 +[2022/04/18 10:17:07] root INFO: name : Momentum +[2022/04/18 10:17:07] root INFO: regularizer : +[2022/04/18 10:17:07] root INFO: coeff : 1e-05 +[2022/04/18 10:17:07] root INFO: name : L2 +[2022/04/18 10:17:07] root INFO: profiler_options : None +[2022/04/18 10:17:07] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:18:29] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:18:29] root INFO: Arch : +[2022/04/18 10:18:29] root INFO: class_num : 102 +[2022/04/18 10:18:29] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:18:29] root INFO: DataLoader : +[2022/04/18 10:18:29] root INFO: Eval : +[2022/04/18 10:18:29] root INFO: dataset : +[2022/04/18 10:18:29] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:18:29] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:18:29] root INFO: name : ImageNetDataset +[2022/04/18 10:18:29] root INFO: transform_ops : +[2022/04/18 10:18:29] root INFO: DecodeImage : +[2022/04/18 10:18:29] root INFO: channel_first : False +[2022/04/18 10:18:29] root INFO: to_rgb : True +[2022/04/18 10:18:29] root INFO: ResizeImage : +[2022/04/18 10:18:29] root INFO: resize_short : 256 +[2022/04/18 10:18:29] root INFO: CropImage : +[2022/04/18 10:18:29] root INFO: size : 224 +[2022/04/18 10:18:29] root INFO: NormalizeImage : +[2022/04/18 10:18:29] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:18:29] root INFO: order : +[2022/04/18 10:18:29] root INFO: scale : 1.0/255.0 +[2022/04/18 10:18:29] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:18:29] root INFO: loader : +[2022/04/18 10:18:29] root INFO: num_workers : 4 +[2022/04/18 10:18:29] root INFO: use_shared_memory : True +[2022/04/18 10:18:29] root INFO: sampler : +[2022/04/18 10:18:29] root INFO: batch_size : 64 +[2022/04/18 10:18:29] root INFO: drop_last : False +[2022/04/18 10:18:29] root INFO: name : DistributedBatchSampler +[2022/04/18 10:18:29] root INFO: shuffle : False +[2022/04/18 10:18:29] root INFO: Train : +[2022/04/18 10:18:29] root INFO: dataset : +[2022/04/18 10:18:29] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:18:29] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:18:29] root INFO: name : ImageNetDataset +[2022/04/18 10:18:29] root INFO: transform_ops : +[2022/04/18 10:18:29] root INFO: DecodeImage : +[2022/04/18 10:18:29] root INFO: channel_first : False +[2022/04/18 10:18:29] root INFO: to_rgb : True +[2022/04/18 10:18:29] root INFO: RandCropImage : +[2022/04/18 10:18:29] root INFO: size : 224 +[2022/04/18 10:18:29] root INFO: RandFlipImage : +[2022/04/18 10:18:29] root INFO: flip_code : 1 +[2022/04/18 10:18:29] root INFO: NormalizeImage : +[2022/04/18 10:18:29] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:18:29] root INFO: order : +[2022/04/18 10:18:29] root INFO: scale : 1.0/255.0 +[2022/04/18 10:18:29] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:18:29] root INFO: loader : +[2022/04/18 10:18:29] root INFO: num_workers : 4 +[2022/04/18 10:18:29] root INFO: use_shared_memory : True +[2022/04/18 10:18:29] root INFO: sampler : +[2022/04/18 10:18:29] root INFO: batch_size : 256 +[2022/04/18 10:18:29] root INFO: drop_last : False +[2022/04/18 10:18:29] root INFO: name : DistributedBatchSampler +[2022/04/18 10:18:29] root INFO: shuffle : True +[2022/04/18 10:18:29] root INFO: Global : +[2022/04/18 10:18:29] root INFO: checkpoints : None +[2022/04/18 10:18:29] root INFO: device : cpu +[2022/04/18 10:18:29] root INFO: epochs : 20 +[2022/04/18 10:18:29] root INFO: eval_during_train : True +[2022/04/18 10:18:29] root INFO: eval_interval : 1 +[2022/04/18 10:18:29] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:18:29] root INFO: output_dir : ./output/ +[2022/04/18 10:18:29] root INFO: pretrained_model : None +[2022/04/18 10:18:29] root INFO: print_batch_step : 10 +[2022/04/18 10:18:29] root INFO: save_inference_dir : ./inference +[2022/04/18 10:18:29] root INFO: save_interval : 1 +[2022/04/18 10:18:29] root INFO: use_visualdl : False +[2022/04/18 10:18:29] root INFO: Infer : +[2022/04/18 10:18:29] root INFO: PostProcess : +[2022/04/18 10:18:29] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:18:29] root INFO: name : Topk +[2022/04/18 10:18:29] root INFO: topk : 5 +[2022/04/18 10:18:29] root INFO: batch_size : 10 +[2022/04/18 10:18:29] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:18:29] root INFO: transforms : +[2022/04/18 10:18:29] root INFO: DecodeImage : +[2022/04/18 10:18:29] root INFO: channel_first : False +[2022/04/18 10:18:29] root INFO: to_rgb : True +[2022/04/18 10:18:29] root INFO: ResizeImage : +[2022/04/18 10:18:29] root INFO: resize_short : 256 +[2022/04/18 10:18:29] root INFO: CropImage : +[2022/04/18 10:18:29] root INFO: size : 224 +[2022/04/18 10:18:29] root INFO: NormalizeImage : +[2022/04/18 10:18:29] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:18:29] root INFO: order : +[2022/04/18 10:18:29] root INFO: scale : 1.0/255.0 +[2022/04/18 10:18:29] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:18:29] root INFO: ToCHWImage : None +[2022/04/18 10:18:29] root INFO: Loss : +[2022/04/18 10:18:29] root INFO: Eval : +[2022/04/18 10:18:29] root INFO: CELoss : +[2022/04/18 10:18:29] root INFO: weight : 1.0 +[2022/04/18 10:18:29] root INFO: Train : +[2022/04/18 10:18:29] root INFO: CELoss : +[2022/04/18 10:18:29] root INFO: weight : 1.0 +[2022/04/18 10:18:29] root INFO: Metric : +[2022/04/18 10:18:29] root INFO: Eval : +[2022/04/18 10:18:29] root INFO: TopkAcc : +[2022/04/18 10:18:29] root INFO: topk : [1, 5] +[2022/04/18 10:18:29] root INFO: Train : +[2022/04/18 10:18:29] root INFO: TopkAcc : +[2022/04/18 10:18:29] root INFO: topk : [1, 5] +[2022/04/18 10:18:29] root INFO: Optimizer : +[2022/04/18 10:18:29] root INFO: lr : +[2022/04/18 10:18:29] root INFO: learning_rate : 0.0125 +[2022/04/18 10:18:29] root INFO: name : Cosine +[2022/04/18 10:18:29] root INFO: warmup_epoch : 5 +[2022/04/18 10:18:29] root INFO: momentum : 0.9 +[2022/04/18 10:18:29] root INFO: name : Momentum +[2022/04/18 10:18:29] root INFO: regularizer : +[2022/04/18 10:18:29] root INFO: coeff : 1e-05 +[2022/04/18 10:18:29] root INFO: name : L2 +[2022/04/18 10:18:29] root INFO: profiler_options : None +[2022/04/18 10:18:29] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:22:55] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:22:55] root INFO: Arch : +[2022/04/18 10:22:55] root INFO: class_num : 102 +[2022/04/18 10:22:55] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:22:55] root INFO: DataLoader : +[2022/04/18 10:22:55] root INFO: Eval : +[2022/04/18 10:22:55] root INFO: dataset : +[2022/04/18 10:22:55] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:22:55] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:22:55] root INFO: name : ImageNetDataset +[2022/04/18 10:22:55] root INFO: transform_ops : +[2022/04/18 10:22:55] root INFO: DecodeImage : +[2022/04/18 10:22:55] root INFO: channel_first : False +[2022/04/18 10:22:55] root INFO: to_rgb : True +[2022/04/18 10:22:55] root INFO: ResizeImage : +[2022/04/18 10:22:55] root INFO: resize_short : 256 +[2022/04/18 10:22:55] root INFO: CropImage : +[2022/04/18 10:22:55] root INFO: size : 224 +[2022/04/18 10:22:55] root INFO: NormalizeImage : +[2022/04/18 10:22:55] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:22:55] root INFO: order : +[2022/04/18 10:22:55] root INFO: scale : 1.0/255.0 +[2022/04/18 10:22:55] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:22:55] root INFO: loader : +[2022/04/18 10:22:55] root INFO: num_workers : 4 +[2022/04/18 10:22:55] root INFO: use_shared_memory : True +[2022/04/18 10:22:55] root INFO: sampler : +[2022/04/18 10:22:55] root INFO: batch_size : 64 +[2022/04/18 10:22:55] root INFO: drop_last : False +[2022/04/18 10:22:55] root INFO: name : DistributedBatchSampler +[2022/04/18 10:22:55] root INFO: shuffle : False +[2022/04/18 10:22:55] root INFO: Train : +[2022/04/18 10:22:55] root INFO: dataset : +[2022/04/18 10:22:55] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:22:55] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:22:55] root INFO: name : ImageNetDataset +[2022/04/18 10:22:55] root INFO: transform_ops : +[2022/04/18 10:22:55] root INFO: DecodeImage : +[2022/04/18 10:22:55] root INFO: channel_first : False +[2022/04/18 10:22:55] root INFO: to_rgb : True +[2022/04/18 10:22:55] root INFO: RandCropImage : +[2022/04/18 10:22:55] root INFO: size : 224 +[2022/04/18 10:22:55] root INFO: RandFlipImage : +[2022/04/18 10:22:55] root INFO: flip_code : 1 +[2022/04/18 10:22:55] root INFO: NormalizeImage : +[2022/04/18 10:22:55] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:22:55] root INFO: order : +[2022/04/18 10:22:55] root INFO: scale : 1.0/255.0 +[2022/04/18 10:22:55] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:22:55] root INFO: loader : +[2022/04/18 10:22:55] root INFO: num_workers : 4 +[2022/04/18 10:22:55] root INFO: use_shared_memory : True +[2022/04/18 10:22:55] root INFO: sampler : +[2022/04/18 10:22:55] root INFO: batch_size : 256 +[2022/04/18 10:22:55] root INFO: drop_last : False +[2022/04/18 10:22:55] root INFO: name : DistributedBatchSampler +[2022/04/18 10:22:55] root INFO: shuffle : True +[2022/04/18 10:22:55] root INFO: Global : +[2022/04/18 10:22:55] root INFO: checkpoints : None +[2022/04/18 10:22:55] root INFO: device : cpu +[2022/04/18 10:22:55] root INFO: epochs : 20 +[2022/04/18 10:22:55] root INFO: eval_during_train : True +[2022/04/18 10:22:55] root INFO: eval_interval : 1 +[2022/04/18 10:22:55] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:22:55] root INFO: output_dir : ./output/ +[2022/04/18 10:22:55] root INFO: pretrained_model : None +[2022/04/18 10:22:55] root INFO: print_batch_step : 10 +[2022/04/18 10:22:55] root INFO: save_inference_dir : ./inference +[2022/04/18 10:22:55] root INFO: save_interval : 1 +[2022/04/18 10:22:55] root INFO: use_visualdl : False +[2022/04/18 10:22:55] root INFO: Infer : +[2022/04/18 10:22:55] root INFO: PostProcess : +[2022/04/18 10:22:55] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:22:55] root INFO: name : Topk +[2022/04/18 10:22:55] root INFO: topk : 5 +[2022/04/18 10:22:55] root INFO: batch_size : 10 +[2022/04/18 10:22:55] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:22:55] root INFO: transforms : +[2022/04/18 10:22:55] root INFO: DecodeImage : +[2022/04/18 10:22:55] root INFO: channel_first : False +[2022/04/18 10:22:55] root INFO: to_rgb : True +[2022/04/18 10:22:55] root INFO: ResizeImage : +[2022/04/18 10:22:55] root INFO: resize_short : 256 +[2022/04/18 10:22:55] root INFO: CropImage : +[2022/04/18 10:22:55] root INFO: size : 224 +[2022/04/18 10:22:55] root INFO: NormalizeImage : +[2022/04/18 10:22:55] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:22:55] root INFO: order : +[2022/04/18 10:22:55] root INFO: scale : 1.0/255.0 +[2022/04/18 10:22:55] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:22:55] root INFO: ToCHWImage : None +[2022/04/18 10:22:55] root INFO: Loss : +[2022/04/18 10:22:55] root INFO: Eval : +[2022/04/18 10:22:55] root INFO: CELoss : +[2022/04/18 10:22:55] root INFO: weight : 1.0 +[2022/04/18 10:22:55] root INFO: Train : +[2022/04/18 10:22:55] root INFO: CELoss : +[2022/04/18 10:22:55] root INFO: weight : 1.0 +[2022/04/18 10:22:55] root INFO: Metric : +[2022/04/18 10:22:55] root INFO: Eval : +[2022/04/18 10:22:55] root INFO: TopkAcc : +[2022/04/18 10:22:55] root INFO: topk : [1, 5] +[2022/04/18 10:22:55] root INFO: Train : +[2022/04/18 10:22:55] root INFO: TopkAcc : +[2022/04/18 10:22:55] root INFO: topk : [1, 5] +[2022/04/18 10:22:55] root INFO: Optimizer : +[2022/04/18 10:22:55] root INFO: lr : +[2022/04/18 10:22:55] root INFO: learning_rate : 0.0125 +[2022/04/18 10:22:55] root INFO: name : Cosine +[2022/04/18 10:22:55] root INFO: warmup_epoch : 5 +[2022/04/18 10:22:55] root INFO: momentum : 0.9 +[2022/04/18 10:22:55] root INFO: name : Momentum +[2022/04/18 10:22:55] root INFO: regularizer : +[2022/04/18 10:22:55] root INFO: coeff : 1e-05 +[2022/04/18 10:22:55] root INFO: name : L2 +[2022/04/18 10:22:55] root INFO: profiler_options : None +[2022/04/18 10:22:55] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:23:40] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:23:40] root INFO: Arch : +[2022/04/18 10:23:40] root INFO: class_num : 102 +[2022/04/18 10:23:40] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:23:40] root INFO: DataLoader : +[2022/04/18 10:23:40] root INFO: Eval : +[2022/04/18 10:23:40] root INFO: dataset : +[2022/04/18 10:23:40] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:23:40] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:23:40] root INFO: name : ImageNetDataset +[2022/04/18 10:23:40] root INFO: transform_ops : +[2022/04/18 10:23:40] root INFO: DecodeImage : +[2022/04/18 10:23:40] root INFO: channel_first : False +[2022/04/18 10:23:40] root INFO: to_rgb : True +[2022/04/18 10:23:40] root INFO: ResizeImage : +[2022/04/18 10:23:40] root INFO: resize_short : 256 +[2022/04/18 10:23:40] root INFO: CropImage : +[2022/04/18 10:23:40] root INFO: size : 224 +[2022/04/18 10:23:40] root INFO: NormalizeImage : +[2022/04/18 10:23:40] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:23:40] root INFO: order : +[2022/04/18 10:23:40] root INFO: scale : 1.0/255.0 +[2022/04/18 10:23:40] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:23:40] root INFO: loader : +[2022/04/18 10:23:40] root INFO: num_workers : 4 +[2022/04/18 10:23:40] root INFO: use_shared_memory : True +[2022/04/18 10:23:40] root INFO: sampler : +[2022/04/18 10:23:40] root INFO: batch_size : 64 +[2022/04/18 10:23:40] root INFO: drop_last : False +[2022/04/18 10:23:40] root INFO: name : DistributedBatchSampler +[2022/04/18 10:23:40] root INFO: shuffle : False +[2022/04/18 10:23:40] root INFO: Train : +[2022/04/18 10:23:40] root INFO: dataset : +[2022/04/18 10:23:40] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:23:40] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:23:40] root INFO: name : ImageNetDataset +[2022/04/18 10:23:40] root INFO: transform_ops : +[2022/04/18 10:23:40] root INFO: DecodeImage : +[2022/04/18 10:23:40] root INFO: channel_first : False +[2022/04/18 10:23:40] root INFO: to_rgb : True +[2022/04/18 10:23:40] root INFO: RandCropImage : +[2022/04/18 10:23:40] root INFO: size : 224 +[2022/04/18 10:23:40] root INFO: RandFlipImage : +[2022/04/18 10:23:40] root INFO: flip_code : 1 +[2022/04/18 10:23:40] root INFO: NormalizeImage : +[2022/04/18 10:23:40] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:23:40] root INFO: order : +[2022/04/18 10:23:40] root INFO: scale : 1.0/255.0 +[2022/04/18 10:23:40] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:23:40] root INFO: loader : +[2022/04/18 10:23:40] root INFO: num_workers : 4 +[2022/04/18 10:23:40] root INFO: use_shared_memory : True +[2022/04/18 10:23:40] root INFO: sampler : +[2022/04/18 10:23:40] root INFO: batch_size : 256 +[2022/04/18 10:23:40] root INFO: drop_last : False +[2022/04/18 10:23:40] root INFO: name : DistributedBatchSampler +[2022/04/18 10:23:40] root INFO: shuffle : True +[2022/04/18 10:23:40] root INFO: Global : +[2022/04/18 10:23:40] root INFO: checkpoints : None +[2022/04/18 10:23:40] root INFO: device : cpu +[2022/04/18 10:23:40] root INFO: epochs : 20 +[2022/04/18 10:23:40] root INFO: eval_during_train : True +[2022/04/18 10:23:40] root INFO: eval_interval : 1 +[2022/04/18 10:23:40] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:23:41] root INFO: output_dir : ./output/ +[2022/04/18 10:23:41] root INFO: pretrained_model : None +[2022/04/18 10:23:41] root INFO: print_batch_step : 10 +[2022/04/18 10:23:41] root INFO: save_inference_dir : ./inference +[2022/04/18 10:23:41] root INFO: save_interval : 1 +[2022/04/18 10:23:41] root INFO: use_visualdl : False +[2022/04/18 10:23:41] root INFO: Infer : +[2022/04/18 10:23:41] root INFO: PostProcess : +[2022/04/18 10:23:41] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:23:41] root INFO: name : Topk +[2022/04/18 10:23:41] root INFO: topk : 5 +[2022/04/18 10:23:41] root INFO: batch_size : 10 +[2022/04/18 10:23:41] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:23:41] root INFO: transforms : +[2022/04/18 10:23:41] root INFO: DecodeImage : +[2022/04/18 10:23:41] root INFO: channel_first : False +[2022/04/18 10:23:41] root INFO: to_rgb : True +[2022/04/18 10:23:41] root INFO: ResizeImage : +[2022/04/18 10:23:41] root INFO: resize_short : 256 +[2022/04/18 10:23:41] root INFO: CropImage : +[2022/04/18 10:23:41] root INFO: size : 224 +[2022/04/18 10:23:41] root INFO: NormalizeImage : +[2022/04/18 10:23:41] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:23:41] root INFO: order : +[2022/04/18 10:23:41] root INFO: scale : 1.0/255.0 +[2022/04/18 10:23:41] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:23:41] root INFO: ToCHWImage : None +[2022/04/18 10:23:41] root INFO: Loss : +[2022/04/18 10:23:41] root INFO: Eval : +[2022/04/18 10:23:41] root INFO: CELoss : +[2022/04/18 10:23:41] root INFO: weight : 1.0 +[2022/04/18 10:23:41] root INFO: Train : +[2022/04/18 10:23:41] root INFO: CELoss : +[2022/04/18 10:23:41] root INFO: weight : 1.0 +[2022/04/18 10:23:41] root INFO: Metric : +[2022/04/18 10:23:41] root INFO: Eval : +[2022/04/18 10:23:41] root INFO: TopkAcc : +[2022/04/18 10:23:41] root INFO: topk : [1, 5] +[2022/04/18 10:23:41] root INFO: Train : +[2022/04/18 10:23:41] root INFO: TopkAcc : +[2022/04/18 10:23:41] root INFO: topk : [1, 5] +[2022/04/18 10:23:41] root INFO: Optimizer : +[2022/04/18 10:23:41] root INFO: lr : +[2022/04/18 10:23:41] root INFO: learning_rate : 0.0125 +[2022/04/18 10:23:41] root INFO: name : Cosine +[2022/04/18 10:23:41] root INFO: warmup_epoch : 5 +[2022/04/18 10:23:41] root INFO: momentum : 0.9 +[2022/04/18 10:23:41] root INFO: name : Momentum +[2022/04/18 10:23:41] root INFO: regularizer : +[2022/04/18 10:23:41] root INFO: coeff : 1e-05 +[2022/04/18 10:23:41] root INFO: name : L2 +[2022/04/18 10:23:41] root INFO: profiler_options : None +[2022/04/18 10:23:41] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:25:40] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:25:40] root INFO: Arch : +[2022/04/18 10:25:40] root INFO: class_num : 102 +[2022/04/18 10:25:40] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:25:40] root INFO: pretrained : True +[2022/04/18 10:25:40] root INFO: DataLoader : +[2022/04/18 10:25:40] root INFO: Eval : +[2022/04/18 10:25:40] root INFO: dataset : +[2022/04/18 10:25:40] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:25:40] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:25:40] root INFO: name : ImageNetDataset +[2022/04/18 10:25:40] root INFO: transform_ops : +[2022/04/18 10:25:40] root INFO: DecodeImage : +[2022/04/18 10:25:40] root INFO: channel_first : False +[2022/04/18 10:25:40] root INFO: to_rgb : True +[2022/04/18 10:25:40] root INFO: ResizeImage : +[2022/04/18 10:25:40] root INFO: resize_short : 256 +[2022/04/18 10:25:40] root INFO: CropImage : +[2022/04/18 10:25:40] root INFO: size : 224 +[2022/04/18 10:25:40] root INFO: NormalizeImage : +[2022/04/18 10:25:40] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:25:40] root INFO: order : +[2022/04/18 10:25:40] root INFO: scale : 1.0/255.0 +[2022/04/18 10:25:40] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:25:40] root INFO: loader : +[2022/04/18 10:25:40] root INFO: num_workers : 4 +[2022/04/18 10:25:40] root INFO: use_shared_memory : True +[2022/04/18 10:25:40] root INFO: sampler : +[2022/04/18 10:25:40] root INFO: batch_size : 64 +[2022/04/18 10:25:40] root INFO: drop_last : False +[2022/04/18 10:25:40] root INFO: name : DistributedBatchSampler +[2022/04/18 10:25:40] root INFO: shuffle : False +[2022/04/18 10:25:40] root INFO: Train : +[2022/04/18 10:25:40] root INFO: dataset : +[2022/04/18 10:25:40] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:25:40] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:25:40] root INFO: name : ImageNetDataset +[2022/04/18 10:25:40] root INFO: transform_ops : +[2022/04/18 10:25:40] root INFO: DecodeImage : +[2022/04/18 10:25:40] root INFO: channel_first : False +[2022/04/18 10:25:40] root INFO: to_rgb : True +[2022/04/18 10:25:40] root INFO: RandCropImage : +[2022/04/18 10:25:40] root INFO: size : 224 +[2022/04/18 10:25:40] root INFO: RandFlipImage : +[2022/04/18 10:25:40] root INFO: flip_code : 1 +[2022/04/18 10:25:40] root INFO: NormalizeImage : +[2022/04/18 10:25:40] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:25:40] root INFO: order : +[2022/04/18 10:25:40] root INFO: scale : 1.0/255.0 +[2022/04/18 10:25:40] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:25:40] root INFO: loader : +[2022/04/18 10:25:40] root INFO: num_workers : 4 +[2022/04/18 10:25:40] root INFO: use_shared_memory : True +[2022/04/18 10:25:40] root INFO: sampler : +[2022/04/18 10:25:40] root INFO: batch_size : 256 +[2022/04/18 10:25:40] root INFO: drop_last : False +[2022/04/18 10:25:40] root INFO: name : DistributedBatchSampler +[2022/04/18 10:25:40] root INFO: shuffle : True +[2022/04/18 10:25:40] root INFO: Global : +[2022/04/18 10:25:40] root INFO: checkpoints : None +[2022/04/18 10:25:40] root INFO: device : cpu +[2022/04/18 10:25:40] root INFO: epochs : 20 +[2022/04/18 10:25:40] root INFO: eval_during_train : True +[2022/04/18 10:25:40] root INFO: eval_interval : 1 +[2022/04/18 10:25:40] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:25:40] root INFO: output_dir : ./output/ +[2022/04/18 10:25:40] root INFO: pretrained_model : None +[2022/04/18 10:25:40] root INFO: print_batch_step : 10 +[2022/04/18 10:25:40] root INFO: save_inference_dir : ./inference +[2022/04/18 10:25:40] root INFO: save_interval : 1 +[2022/04/18 10:25:40] root INFO: use_visualdl : False +[2022/04/18 10:25:40] root INFO: Infer : +[2022/04/18 10:25:40] root INFO: PostProcess : +[2022/04/18 10:25:40] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:25:40] root INFO: name : Topk +[2022/04/18 10:25:40] root INFO: topk : 5 +[2022/04/18 10:25:40] root INFO: batch_size : 10 +[2022/04/18 10:25:40] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:25:40] root INFO: transforms : +[2022/04/18 10:25:40] root INFO: DecodeImage : +[2022/04/18 10:25:40] root INFO: channel_first : False +[2022/04/18 10:25:40] root INFO: to_rgb : True +[2022/04/18 10:25:40] root INFO: ResizeImage : +[2022/04/18 10:25:40] root INFO: resize_short : 256 +[2022/04/18 10:25:40] root INFO: CropImage : +[2022/04/18 10:25:40] root INFO: size : 224 +[2022/04/18 10:25:40] root INFO: NormalizeImage : +[2022/04/18 10:25:40] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:25:40] root INFO: order : +[2022/04/18 10:25:40] root INFO: scale : 1.0/255.0 +[2022/04/18 10:25:40] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:25:40] root INFO: ToCHWImage : None +[2022/04/18 10:25:40] root INFO: Loss : +[2022/04/18 10:25:40] root INFO: Eval : +[2022/04/18 10:25:40] root INFO: CELoss : +[2022/04/18 10:25:40] root INFO: weight : 1.0 +[2022/04/18 10:25:40] root INFO: Train : +[2022/04/18 10:25:40] root INFO: CELoss : +[2022/04/18 10:25:40] root INFO: weight : 1.0 +[2022/04/18 10:25:40] root INFO: Metric : +[2022/04/18 10:25:40] root INFO: Eval : +[2022/04/18 10:25:40] root INFO: TopkAcc : +[2022/04/18 10:25:40] root INFO: topk : [1, 5] +[2022/04/18 10:25:40] root INFO: Train : +[2022/04/18 10:25:40] root INFO: TopkAcc : +[2022/04/18 10:25:40] root INFO: topk : [1, 5] +[2022/04/18 10:25:40] root INFO: Optimizer : +[2022/04/18 10:25:40] root INFO: lr : +[2022/04/18 10:25:40] root INFO: learning_rate : 0.0125 +[2022/04/18 10:25:40] root INFO: name : Cosine +[2022/04/18 10:25:40] root INFO: warmup_epoch : 5 +[2022/04/18 10:25:40] root INFO: momentum : 0.9 +[2022/04/18 10:25:40] root INFO: name : Momentum +[2022/04/18 10:25:40] root INFO: regularizer : +[2022/04/18 10:25:40] root INFO: coeff : 1e-05 +[2022/04/18 10:25:40] root INFO: name : L2 +[2022/04/18 10:25:40] root INFO: profiler_options : None +[2022/04/18 10:25:40] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:27:41] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:27:41] root INFO: Arch : +[2022/04/18 10:27:41] root INFO: class_num : 102 +[2022/04/18 10:27:41] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:27:41] root INFO: DataLoader : +[2022/04/18 10:27:41] root INFO: Eval : +[2022/04/18 10:27:41] root INFO: dataset : +[2022/04/18 10:27:41] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:27:41] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:27:41] root INFO: name : ImageNetDataset +[2022/04/18 10:27:41] root INFO: transform_ops : +[2022/04/18 10:27:41] root INFO: DecodeImage : +[2022/04/18 10:27:41] root INFO: channel_first : False +[2022/04/18 10:27:41] root INFO: to_rgb : True +[2022/04/18 10:27:41] root INFO: ResizeImage : +[2022/04/18 10:27:41] root INFO: resize_short : 256 +[2022/04/18 10:27:41] root INFO: CropImage : +[2022/04/18 10:27:41] root INFO: size : 224 +[2022/04/18 10:27:41] root INFO: NormalizeImage : +[2022/04/18 10:27:41] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:27:41] root INFO: order : +[2022/04/18 10:27:41] root INFO: scale : 1.0/255.0 +[2022/04/18 10:27:41] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:27:41] root INFO: loader : +[2022/04/18 10:27:41] root INFO: num_workers : 4 +[2022/04/18 10:27:41] root INFO: use_shared_memory : True +[2022/04/18 10:27:41] root INFO: sampler : +[2022/04/18 10:27:41] root INFO: batch_size : 64 +[2022/04/18 10:27:41] root INFO: drop_last : False +[2022/04/18 10:27:41] root INFO: name : DistributedBatchSampler +[2022/04/18 10:27:41] root INFO: shuffle : False +[2022/04/18 10:27:41] root INFO: Train : +[2022/04/18 10:27:41] root INFO: dataset : +[2022/04/18 10:27:41] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:27:41] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:27:41] root INFO: name : ImageNetDataset +[2022/04/18 10:27:41] root INFO: transform_ops : +[2022/04/18 10:27:41] root INFO: DecodeImage : +[2022/04/18 10:27:41] root INFO: channel_first : False +[2022/04/18 10:27:41] root INFO: to_rgb : True +[2022/04/18 10:27:41] root INFO: RandCropImage : +[2022/04/18 10:27:41] root INFO: size : 224 +[2022/04/18 10:27:41] root INFO: RandFlipImage : +[2022/04/18 10:27:41] root INFO: flip_code : 1 +[2022/04/18 10:27:41] root INFO: NormalizeImage : +[2022/04/18 10:27:41] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:27:41] root INFO: order : +[2022/04/18 10:27:41] root INFO: scale : 1.0/255.0 +[2022/04/18 10:27:41] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:27:41] root INFO: loader : +[2022/04/18 10:27:41] root INFO: num_workers : 4 +[2022/04/18 10:27:41] root INFO: use_shared_memory : True +[2022/04/18 10:27:41] root INFO: sampler : +[2022/04/18 10:27:41] root INFO: batch_size : 256 +[2022/04/18 10:27:41] root INFO: drop_last : False +[2022/04/18 10:27:41] root INFO: name : DistributedBatchSampler +[2022/04/18 10:27:41] root INFO: shuffle : True +[2022/04/18 10:27:41] root INFO: Global : +[2022/04/18 10:27:41] root INFO: checkpoints : None +[2022/04/18 10:27:41] root INFO: device : cpu +[2022/04/18 10:27:41] root INFO: epochs : 20 +[2022/04/18 10:27:41] root INFO: eval_during_train : True +[2022/04/18 10:27:41] root INFO: eval_interval : 1 +[2022/04/18 10:27:41] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:27:41] root INFO: output_dir : ./output/ +[2022/04/18 10:27:41] root INFO: pretrained_model : None +[2022/04/18 10:27:41] root INFO: print_batch_step : 10 +[2022/04/18 10:27:41] root INFO: save_inference_dir : ./inference +[2022/04/18 10:27:41] root INFO: save_interval : 1 +[2022/04/18 10:27:41] root INFO: use_visualdl : False +[2022/04/18 10:27:41] root INFO: Infer : +[2022/04/18 10:27:41] root INFO: PostProcess : +[2022/04/18 10:27:41] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:27:41] root INFO: name : Topk +[2022/04/18 10:27:41] root INFO: topk : 5 +[2022/04/18 10:27:41] root INFO: batch_size : 10 +[2022/04/18 10:27:41] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:27:41] root INFO: transforms : +[2022/04/18 10:27:41] root INFO: DecodeImage : +[2022/04/18 10:27:41] root INFO: channel_first : False +[2022/04/18 10:27:41] root INFO: to_rgb : True +[2022/04/18 10:27:41] root INFO: ResizeImage : +[2022/04/18 10:27:41] root INFO: resize_short : 256 +[2022/04/18 10:27:41] root INFO: CropImage : +[2022/04/18 10:27:41] root INFO: size : 224 +[2022/04/18 10:27:41] root INFO: NormalizeImage : +[2022/04/18 10:27:41] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:27:41] root INFO: order : +[2022/04/18 10:27:41] root INFO: scale : 1.0/255.0 +[2022/04/18 10:27:41] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:27:41] root INFO: ToCHWImage : None +[2022/04/18 10:27:41] root INFO: Loss : +[2022/04/18 10:27:41] root INFO: Eval : +[2022/04/18 10:27:41] root INFO: CELoss : +[2022/04/18 10:27:41] root INFO: weight : 1.0 +[2022/04/18 10:27:41] root INFO: Train : +[2022/04/18 10:27:41] root INFO: CELoss : +[2022/04/18 10:27:41] root INFO: weight : 1.0 +[2022/04/18 10:27:41] root INFO: Metric : +[2022/04/18 10:27:41] root INFO: Eval : +[2022/04/18 10:27:41] root INFO: TopkAcc : +[2022/04/18 10:27:41] root INFO: topk : [1, 5] +[2022/04/18 10:27:41] root INFO: Train : +[2022/04/18 10:27:41] root INFO: TopkAcc : +[2022/04/18 10:27:41] root INFO: topk : [1, 5] +[2022/04/18 10:27:41] root INFO: Optimizer : +[2022/04/18 10:27:41] root INFO: lr : +[2022/04/18 10:27:41] root INFO: learning_rate : 0.0125 +[2022/04/18 10:27:41] root INFO: name : Cosine +[2022/04/18 10:27:41] root INFO: warmup_epoch : 5 +[2022/04/18 10:27:41] root INFO: momentum : 0.9 +[2022/04/18 10:27:41] root INFO: name : Momentum +[2022/04/18 10:27:41] root INFO: regularizer : +[2022/04/18 10:27:41] root INFO: coeff : 1e-05 +[2022/04/18 10:27:41] root INFO: name : L2 +[2022/04/18 10:27:41] root INFO: profiler_options : None +[2022/04/18 10:27:41] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:29:50] root INFO: +=========================================================== +== PaddleClas is powered by PaddlePaddle ! == +=========================================================== +== == +== For more info please go to the following website. == +== == +== https://github.com/PaddlePaddle/PaddleClas == +=========================================================== + +[2022/04/18 10:29:50] root INFO: Arch : +[2022/04/18 10:29:50] root INFO: class_num : 102 +[2022/04/18 10:29:50] root INFO: name : ShuffleNetV2_x0_25 +[2022/04/18 10:29:50] root INFO: DataLoader : +[2022/04/18 10:29:50] root INFO: Eval : +[2022/04/18 10:29:50] root INFO: dataset : +[2022/04/18 10:29:50] root INFO: cls_label_path : ./dataset/flowers102/val_list.txt +[2022/04/18 10:29:50] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:29:50] root INFO: name : ImageNetDataset +[2022/04/18 10:29:50] root INFO: transform_ops : +[2022/04/18 10:29:50] root INFO: DecodeImage : +[2022/04/18 10:29:50] root INFO: channel_first : False +[2022/04/18 10:29:50] root INFO: to_rgb : True +[2022/04/18 10:29:50] root INFO: ResizeImage : +[2022/04/18 10:29:50] root INFO: resize_short : 256 +[2022/04/18 10:29:50] root INFO: CropImage : +[2022/04/18 10:29:50] root INFO: size : 224 +[2022/04/18 10:29:50] root INFO: NormalizeImage : +[2022/04/18 10:29:50] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:29:50] root INFO: order : +[2022/04/18 10:29:50] root INFO: scale : 1.0/255.0 +[2022/04/18 10:29:50] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:29:50] root INFO: loader : +[2022/04/18 10:29:50] root INFO: num_workers : 4 +[2022/04/18 10:29:50] root INFO: use_shared_memory : True +[2022/04/18 10:29:50] root INFO: sampler : +[2022/04/18 10:29:50] root INFO: batch_size : 64 +[2022/04/18 10:29:50] root INFO: drop_last : False +[2022/04/18 10:29:50] root INFO: name : DistributedBatchSampler +[2022/04/18 10:29:50] root INFO: shuffle : False +[2022/04/18 10:29:50] root INFO: Train : +[2022/04/18 10:29:50] root INFO: dataset : +[2022/04/18 10:29:50] root INFO: cls_label_path : ./dataset/flowers102/train_list.txt +[2022/04/18 10:29:50] root INFO: image_root : ./dataset/flowers102/ +[2022/04/18 10:29:50] root INFO: name : ImageNetDataset +[2022/04/18 10:29:50] root INFO: transform_ops : +[2022/04/18 10:29:50] root INFO: DecodeImage : +[2022/04/18 10:29:50] root INFO: channel_first : False +[2022/04/18 10:29:50] root INFO: to_rgb : True +[2022/04/18 10:29:50] root INFO: RandCropImage : +[2022/04/18 10:29:50] root INFO: size : 224 +[2022/04/18 10:29:50] root INFO: RandFlipImage : +[2022/04/18 10:29:50] root INFO: flip_code : 1 +[2022/04/18 10:29:50] root INFO: NormalizeImage : +[2022/04/18 10:29:50] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:29:50] root INFO: order : +[2022/04/18 10:29:50] root INFO: scale : 1.0/255.0 +[2022/04/18 10:29:50] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:29:50] root INFO: loader : +[2022/04/18 10:29:50] root INFO: num_workers : 4 +[2022/04/18 10:29:50] root INFO: use_shared_memory : True +[2022/04/18 10:29:50] root INFO: sampler : +[2022/04/18 10:29:50] root INFO: batch_size : 256 +[2022/04/18 10:29:50] root INFO: drop_last : False +[2022/04/18 10:29:50] root INFO: name : DistributedBatchSampler +[2022/04/18 10:29:50] root INFO: shuffle : True +[2022/04/18 10:29:50] root INFO: Global : +[2022/04/18 10:29:50] root INFO: checkpoints : None +[2022/04/18 10:29:50] root INFO: device : cpu +[2022/04/18 10:29:50] root INFO: epochs : 20 +[2022/04/18 10:29:50] root INFO: eval_during_train : True +[2022/04/18 10:29:50] root INFO: eval_interval : 1 +[2022/04/18 10:29:50] root INFO: image_shape : [3, 224, 224] +[2022/04/18 10:29:50] root INFO: output_dir : ./output/ +[2022/04/18 10:29:50] root INFO: pretrained_model : None +[2022/04/18 10:29:50] root INFO: print_batch_step : 10 +[2022/04/18 10:29:50] root INFO: save_inference_dir : ./inference +[2022/04/18 10:29:50] root INFO: save_interval : 1 +[2022/04/18 10:29:50] root INFO: use_visualdl : False +[2022/04/18 10:29:50] root INFO: Infer : +[2022/04/18 10:29:50] root INFO: PostProcess : +[2022/04/18 10:29:50] root INFO: class_id_map_file : ./dataset/flowers102/flowers102_label_list.txt +[2022/04/18 10:29:50] root INFO: name : Topk +[2022/04/18 10:29:50] root INFO: topk : 5 +[2022/04/18 10:29:50] root INFO: batch_size : 10 +[2022/04/18 10:29:50] root INFO: infer_imgs : docs/images/inference_deployment/whl_demo.jpg +[2022/04/18 10:29:50] root INFO: transforms : +[2022/04/18 10:29:50] root INFO: DecodeImage : +[2022/04/18 10:29:50] root INFO: channel_first : False +[2022/04/18 10:29:50] root INFO: to_rgb : True +[2022/04/18 10:29:50] root INFO: ResizeImage : +[2022/04/18 10:29:50] root INFO: resize_short : 256 +[2022/04/18 10:29:50] root INFO: CropImage : +[2022/04/18 10:29:50] root INFO: size : 224 +[2022/04/18 10:29:50] root INFO: NormalizeImage : +[2022/04/18 10:29:50] root INFO: mean : [0.485, 0.456, 0.406] +[2022/04/18 10:29:50] root INFO: order : +[2022/04/18 10:29:50] root INFO: scale : 1.0/255.0 +[2022/04/18 10:29:50] root INFO: std : [0.229, 0.224, 0.225] +[2022/04/18 10:29:50] root INFO: ToCHWImage : None +[2022/04/18 10:29:50] root INFO: Loss : +[2022/04/18 10:29:50] root INFO: Eval : +[2022/04/18 10:29:50] root INFO: CELoss : +[2022/04/18 10:29:50] root INFO: weight : 1.0 +[2022/04/18 10:29:50] root INFO: Train : +[2022/04/18 10:29:50] root INFO: CELoss : +[2022/04/18 10:29:50] root INFO: weight : 1.0 +[2022/04/18 10:29:50] root INFO: Metric : +[2022/04/18 10:29:50] root INFO: Eval : +[2022/04/18 10:29:50] root INFO: TopkAcc : +[2022/04/18 10:29:50] root INFO: topk : [1, 5] +[2022/04/18 10:29:50] root INFO: Train : +[2022/04/18 10:29:50] root INFO: TopkAcc : +[2022/04/18 10:29:50] root INFO: topk : [1, 5] +[2022/04/18 10:29:50] root INFO: Optimizer : +[2022/04/18 10:29:50] root INFO: lr : +[2022/04/18 10:29:50] root INFO: learning_rate : 0.0125 +[2022/04/18 10:29:50] root INFO: name : Cosine +[2022/04/18 10:29:50] root INFO: warmup_epoch : 5 +[2022/04/18 10:29:50] root INFO: momentum : 0.9 +[2022/04/18 10:29:50] root INFO: name : Momentum +[2022/04/18 10:29:50] root INFO: regularizer : +[2022/04/18 10:29:50] root INFO: coeff : 1e-05 +[2022/04/18 10:29:50] root INFO: name : L2 +[2022/04/18 10:29:50] root INFO: profiler_options : None +[2022/04/18 10:29:50] root INFO: train with paddle 2.2.2 and device CPUPlace +[2022/04/18 10:29:50] root WARNING: The training strategy in config files provided by PaddleClas is based on 4 gpus. But the number of gpus is 1 in current training. Please modify the stategy (learning rate, batch size and so on) if use config files in PaddleClas to train. +[2022/04/18 10:30:01] root INFO: [Train][Epoch 1/20][Iter: 0/4]lr: 0.00063, top1: 0.01562, top5: 0.04688, CELoss: 4.72476, loss: 4.72476, batch_cost: 11.32137s, reader_cost: 1.80961, ips: 22.61211 images/sec, eta: 0:15:05 +[2022/04/18 10:30:12] root INFO: [Train][Epoch 1/20][Avg]top1: 0.01823, top5: 0.05859, CELoss: 4.74742, loss: 4.74742 +[2022/04/18 10:30:12] root INFO: [Eval][Epoch 1][Iter: 0/16]CELoss: 7.71317, loss: 7.71317, top1: 0.00000, top5: 0.00000, batch_cost: 0.87117s, reader_cost: 0.50988, ips: 73.46428 images/sec +[2022/04/18 10:30:18] root INFO: [Eval][Epoch 1][Iter: 10/16]CELoss: 9.23040, loss: 9.23040, top1: 0.00000, top5: 0.06250, batch_cost: 0.52336s, reader_cost: 0.16568, ips: 122.28759 images/sec +[2022/04/18 10:30:20] root INFO: [Eval][Epoch 1][Avg]CELoss: 8.66056, loss: 8.66056, top1: 0.01042, top5: 0.05208 +[2022/04/18 10:30:20] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\best_model +[2022/04/18 10:30:20] root INFO: [Eval][Epoch 1][best metric: 0.010416666666666666] +[2022/04/18 10:30:20] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_1 +[2022/04/18 10:30:20] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:30:27] root INFO: [Train][Epoch 2/20][Iter: 0/4]lr: 0.00250, top1: 0.01562, top5: 0.06641, CELoss: 4.73765, loss: 4.73765, batch_cost: 7.19691s, reader_cost: 0.89064, ips: 35.57083 images/sec, eta: 0:09:06 +[2022/04/18 10:30:38] root INFO: [Train][Epoch 2/20][Avg]top1: 0.01823, top5: 0.05469, CELoss: 4.75268, loss: 4.75268 +[2022/04/18 10:30:39] root INFO: [Eval][Epoch 2][Iter: 0/16]CELoss: 4.73498, loss: 4.73498, top1: 0.01562, top5: 0.07812, batch_cost: 0.80981s, reader_cost: 0.44976, ips: 79.03089 images/sec +[2022/04/18 10:30:44] root INFO: [Eval][Epoch 2][Iter: 10/16]CELoss: 4.71561, loss: 4.71561, top1: 0.04688, top5: 0.20312, batch_cost: 0.50151s, reader_cost: 0.13841, ips: 127.61419 images/sec +[2022/04/18 10:30:46] root INFO: [Eval][Epoch 2][Avg]CELoss: 4.86986, loss: 4.86986, top1: 0.01354, top5: 0.05729 +[2022/04/18 10:30:46] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\best_model +[2022/04/18 10:30:46] root INFO: [Eval][Epoch 2][best metric: 0.013541666666666667] +[2022/04/18 10:30:46] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_2 +[2022/04/18 10:30:46] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:30:53] root INFO: [Train][Epoch 3/20][Iter: 0/4]lr: 0.00438, top1: 0.02344, top5: 0.07031, CELoss: 4.72546, loss: 4.72546, batch_cost: 6.63502s, reader_cost: 0.73936, ips: 38.58317 images/sec, eta: 0:07:57 +[2022/04/18 10:31:03] root INFO: [Train][Epoch 3/20][Avg]top1: 0.01693, top5: 0.06380, CELoss: 4.74237, loss: 4.74237 +[2022/04/18 10:31:04] root INFO: [Eval][Epoch 3][Iter: 0/16]CELoss: 4.63404, loss: 4.63404, top1: 0.00000, top5: 0.12500, batch_cost: 0.82113s, reader_cost: 0.44998, ips: 77.94178 images/sec +[2022/04/18 10:31:09] root INFO: [Eval][Epoch 3][Iter: 10/16]CELoss: 4.44187, loss: 4.44187, top1: 0.14062, top5: 0.39062, batch_cost: 0.50167s, reader_cost: 0.13407, ips: 127.57276 images/sec +[2022/04/18 10:31:11] root INFO: [Eval][Epoch 3][Avg]CELoss: 4.66967, loss: 4.66967, top1: 0.01042, top5: 0.05104 +[2022/04/18 10:31:11] root INFO: [Eval][Epoch 3][best metric: 0.013541666666666667] +[2022/04/18 10:31:11] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_3 +[2022/04/18 10:31:11] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:31:20] root INFO: [Train][Epoch 4/20][Iter: 0/4]lr: 0.00625, top1: 0.01172, top5: 0.05078, CELoss: 4.74420, loss: 4.74420, batch_cost: 6.52034s, reader_cost: 0.68773, ips: 39.26178 images/sec, eta: 0:07:23 +[2022/04/18 10:31:31] root INFO: [Train][Epoch 4/20][Avg]top1: 0.01432, top5: 0.05729, CELoss: 4.73518, loss: 4.73518 +[2022/04/18 10:31:32] root INFO: [Eval][Epoch 4][Iter: 0/16]CELoss: 4.60896, loss: 4.60896, top1: 0.04688, top5: 0.12500, batch_cost: 0.92905s, reader_cost: 0.51022, ips: 68.88763 images/sec +[2022/04/18 10:31:38] root INFO: [Eval][Epoch 4][Iter: 10/16]CELoss: 4.42349, loss: 4.42349, top1: 0.01562, top5: 0.34375, batch_cost: 0.58857s, reader_cost: 0.15611, ips: 108.73772 images/sec +[2022/04/18 10:31:40] root INFO: [Eval][Epoch 4][Avg]CELoss: 4.65612, loss: 4.65612, top1: 0.00833, top5: 0.05208 +[2022/04/18 10:31:40] root INFO: [Eval][Epoch 4][best metric: 0.013541666666666667] +[2022/04/18 10:31:40] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_4 +[2022/04/18 10:31:40] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:31:49] root INFO: [Train][Epoch 5/20][Iter: 0/4]lr: 0.00813, top1: 0.03125, top5: 0.08203, CELoss: 4.62534, loss: 4.62534, batch_cost: 6.55339s, reader_cost: 0.65519, ips: 39.06375 images/sec, eta: 0:06:59 +[2022/04/18 10:32:02] root INFO: [Train][Epoch 5/20][Avg]top1: 0.01562, top5: 0.05729, CELoss: 4.68708, loss: 4.68708 +[2022/04/18 10:32:02] root INFO: [Eval][Epoch 5][Iter: 0/16]CELoss: 4.60583, loss: 4.60583, top1: 0.04688, top5: 0.12500, batch_cost: 0.98133s, reader_cost: 0.50026, ips: 65.21790 images/sec +[2022/04/18 10:32:09] root INFO: [Eval][Epoch 5][Iter: 10/16]CELoss: 4.42307, loss: 4.42307, top1: 0.03125, top5: 0.40625, batch_cost: 0.65791s, reader_cost: 0.18276, ips: 97.27800 images/sec +[2022/04/18 10:32:11] root INFO: [Eval][Epoch 5][Avg]CELoss: 4.64909, loss: 4.64909, top1: 0.01146, top5: 0.06250 +[2022/04/18 10:32:11] root INFO: [Eval][Epoch 5][best metric: 0.013541666666666667] +[2022/04/18 10:32:11] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_5 +[2022/04/18 10:32:11] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:32:20] root INFO: [Train][Epoch 6/20][Iter: 0/4]lr: 0.01000, top1: 0.01172, top5: 0.04297, CELoss: 4.68610, loss: 4.68610, batch_cost: 6.63889s, reader_cost: 0.63984, ips: 38.56064 images/sec, eta: 0:06:38 +[2022/04/18 10:32:31] root INFO: [Train][Epoch 6/20][Avg]top1: 0.01172, top5: 0.06250, CELoss: 4.68376, loss: 4.68376 +[2022/04/18 10:32:32] root INFO: [Eval][Epoch 6][Iter: 0/16]CELoss: 4.61412, loss: 4.61412, top1: 0.03125, top5: 0.06250, batch_cost: 0.86996s, reader_cost: 0.43996, ips: 73.56664 images/sec +[2022/04/18 10:32:38] root INFO: [Eval][Epoch 6][Iter: 10/16]CELoss: 4.44697, loss: 4.44697, top1: 0.03125, top5: 0.34375, batch_cost: 0.57997s, reader_cost: 0.15333, ips: 110.34973 images/sec +[2022/04/18 10:32:40] root INFO: [Eval][Epoch 6][Avg]CELoss: 4.64477, loss: 4.64477, top1: 0.00937, top5: 0.05000 +[2022/04/18 10:32:40] root INFO: [Eval][Epoch 6][best metric: 0.013541666666666667] +[2022/04/18 10:32:40] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_6 +[2022/04/18 10:32:40] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:32:48] root INFO: [Train][Epoch 7/20][Iter: 0/4]lr: 0.01188, top1: 0.01562, top5: 0.07031, CELoss: 4.65131, loss: 4.65131, batch_cost: 6.59630s, reader_cost: 0.62655, ips: 38.80962 images/sec, eta: 0:06:09 +[2022/04/18 10:32:58] root INFO: [Train][Epoch 7/20][Avg]top1: 0.01172, top5: 0.06510, CELoss: 4.65130, loss: 4.65130 +[2022/04/18 10:32:59] root INFO: [Eval][Epoch 7][Iter: 0/16]CELoss: 4.61308, loss: 4.61308, top1: 0.03125, top5: 0.03125, batch_cost: 0.87047s, reader_cost: 0.46899, ips: 73.52333 images/sec +[2022/04/18 10:33:05] root INFO: [Eval][Epoch 7][Iter: 10/16]CELoss: 4.49343, loss: 4.49343, top1: 0.07812, top5: 0.23438, batch_cost: 0.55668s, reader_cost: 0.14980, ips: 114.96690 images/sec +[2022/04/18 10:33:07] root INFO: [Eval][Epoch 7][Avg]CELoss: 4.63089, loss: 4.63089, top1: 0.01354, top5: 0.05625 +[2022/04/18 10:33:07] root INFO: [Eval][Epoch 7][best metric: 0.013541666666666667] +[2022/04/18 10:33:07] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_7 +[2022/04/18 10:33:07] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:33:16] root INFO: [Train][Epoch 8/20][Iter: 0/4]lr: 0.01247, top1: 0.00781, top5: 0.05469, CELoss: 4.64280, loss: 4.64280, batch_cost: 6.53036s, reader_cost: 0.61650, ips: 39.20150 images/sec, eta: 0:05:39 +[2022/04/18 10:33:25] root INFO: [Train][Epoch 8/20][Avg]top1: 0.01042, top5: 0.04557, CELoss: 4.64303, loss: 4.64303 +[2022/04/18 10:33:26] root INFO: [Eval][Epoch 8][Iter: 0/16]CELoss: 4.62822, loss: 4.62822, top1: 0.00000, top5: 0.01562, batch_cost: 0.84976s, reader_cost: 0.44264, ips: 75.31527 images/sec +[2022/04/18 10:33:32] root INFO: [Eval][Epoch 8][Iter: 10/16]CELoss: 4.52708, loss: 4.52708, top1: 0.03125, top5: 0.14062, batch_cost: 0.55012s, reader_cost: 0.15373, ips: 116.33885 images/sec +[2022/04/18 10:33:34] root INFO: [Eval][Epoch 8][Avg]CELoss: 4.62241, loss: 4.62241, top1: 0.00833, top5: 0.05521 +[2022/04/18 10:33:34] root INFO: [Eval][Epoch 8][best metric: 0.013541666666666667] +[2022/04/18 10:33:34] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_8 +[2022/04/18 10:33:34] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:33:42] root INFO: [Train][Epoch 9/20][Iter: 0/4]lr: 0.01229, top1: 0.02734, top5: 0.08203, CELoss: 4.62807, loss: 4.62807, batch_cost: 6.46556s, reader_cost: 0.60792, ips: 39.59440 images/sec, eta: 0:05:10 +[2022/04/18 10:33:52] root INFO: [Train][Epoch 9/20][Avg]top1: 0.01172, top5: 0.06250, CELoss: 4.61966, loss: 4.61966 +[2022/04/18 10:33:53] root INFO: [Eval][Epoch 9][Iter: 0/16]CELoss: 4.65428, loss: 4.65428, top1: 0.00000, top5: 0.00000, batch_cost: 0.81872s, reader_cost: 0.44007, ips: 78.17127 images/sec +[2022/04/18 10:33:59] root INFO: [Eval][Epoch 9][Iter: 10/16]CELoss: 4.58036, loss: 4.58036, top1: 0.04688, top5: 0.09375, batch_cost: 0.55475s, reader_cost: 0.15056, ips: 115.36788 images/sec +[2022/04/18 10:34:01] root INFO: [Eval][Epoch 9][Avg]CELoss: 4.61833, loss: 4.61833, top1: 0.01354, top5: 0.05417 +[2022/04/18 10:34:01] root INFO: [Eval][Epoch 9][best metric: 0.013541666666666667] +[2022/04/18 10:34:01] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_9 +[2022/04/18 10:34:01] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:34:09] root INFO: [Train][Epoch 10/20][Iter: 0/4]lr: 0.01196, top1: 0.01953, top5: 0.05469, CELoss: 4.60297, loss: 4.60297, batch_cost: 6.43565s, reader_cost: 0.61064, ips: 39.77840 images/sec, eta: 0:04:43 +[2022/04/18 10:34:19] root INFO: [Train][Epoch 10/20][Avg]top1: 0.01953, top5: 0.07422, CELoss: 4.60467, loss: 4.60467 +[2022/04/18 10:34:20] root INFO: [Eval][Epoch 10][Iter: 0/16]CELoss: 4.66085, loss: 4.66085, top1: 0.00000, top5: 0.06250, batch_cost: 0.82531s, reader_cost: 0.44567, ips: 77.54660 images/sec +[2022/04/18 10:34:26] root INFO: [Eval][Epoch 10][Iter: 10/16]CELoss: 4.61584, loss: 4.61584, top1: 0.00000, top5: 0.03125, batch_cost: 0.54999s, reader_cost: 0.15064, ips: 116.36524 images/sec +[2022/04/18 10:34:28] root INFO: [Eval][Epoch 10][Avg]CELoss: 4.61231, loss: 4.61231, top1: 0.00729, top5: 0.05417 +[2022/04/18 10:34:28] root INFO: [Eval][Epoch 10][best metric: 0.013541666666666667] +[2022/04/18 10:34:28] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_10 +[2022/04/18 10:34:28] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:34:36] root INFO: [Train][Epoch 11/20][Iter: 0/4]lr: 0.01149, top1: 0.00781, top5: 0.03906, CELoss: 4.62249, loss: 4.62249, batch_cost: 6.38626s, reader_cost: 0.60446, ips: 40.08603 images/sec, eta: 0:04:15 +[2022/04/18 10:34:46] root INFO: [Train][Epoch 11/20][Avg]top1: 0.01042, top5: 0.06120, CELoss: 4.61070, loss: 4.61070 +[2022/04/18 10:34:47] root INFO: [Eval][Epoch 11][Iter: 0/16]CELoss: 4.64748, loss: 4.64748, top1: 0.00000, top5: 0.03125, batch_cost: 0.86118s, reader_cost: 0.44960, ips: 74.31663 images/sec +[2022/04/18 10:34:53] root INFO: [Eval][Epoch 11][Iter: 10/16]CELoss: 4.61273, loss: 4.61273, top1: 0.00000, top5: 0.04688, batch_cost: 0.58394s, reader_cost: 0.16238, ips: 109.59963 images/sec +[2022/04/18 10:34:55] root INFO: [Eval][Epoch 11][Avg]CELoss: 4.60653, loss: 4.60653, top1: 0.01562, top5: 0.06667 +[2022/04/18 10:34:55] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\best_model +[2022/04/18 10:34:55] root INFO: [Eval][Epoch 11][best metric: 0.015625] +[2022/04/18 10:34:55] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_11 +[2022/04/18 10:34:55] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:35:03] root INFO: [Train][Epoch 12/20][Iter: 0/4]lr: 0.01089, top1: 0.00781, top5: 0.05859, CELoss: 4.60968, loss: 4.60968, batch_cost: 6.35985s, reader_cost: 0.60053, ips: 40.25252 images/sec, eta: 0:03:48 +[2022/04/18 10:35:13] root INFO: [Train][Epoch 12/20][Avg]top1: 0.00781, top5: 0.06901, CELoss: 4.60008, loss: 4.60008 +[2022/04/18 10:35:14] root INFO: [Eval][Epoch 12][Iter: 0/16]CELoss: 4.66432, loss: 4.66432, top1: 0.00000, top5: 0.01562, batch_cost: 0.84842s, reader_cost: 0.44881, ips: 75.43432 images/sec +[2022/04/18 10:35:19] root INFO: [Eval][Epoch 12][Iter: 10/16]CELoss: 4.60226, loss: 4.60226, top1: 0.01562, top5: 0.03125, batch_cost: 0.54736s, reader_cost: 0.14973, ips: 116.92520 images/sec +[2022/04/18 10:35:22] root INFO: [Eval][Epoch 12][Avg]CELoss: 4.59884, loss: 4.59884, top1: 0.01250, top5: 0.05937 +[2022/04/18 10:35:22] root INFO: [Eval][Epoch 12][best metric: 0.015625] +[2022/04/18 10:35:22] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_12 +[2022/04/18 10:35:22] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:35:30] root INFO: [Train][Epoch 13/20][Iter: 0/4]lr: 0.01018, top1: 0.00781, top5: 0.07031, CELoss: 4.58294, loss: 4.58294, batch_cost: 6.33882s, reader_cost: 0.59699, ips: 40.38606 images/sec, eta: 0:03:22 +[2022/04/18 10:35:40] root INFO: [Train][Epoch 13/20][Avg]top1: 0.00781, top5: 0.07292, CELoss: 4.59903, loss: 4.59903 +[2022/04/18 10:35:41] root INFO: [Eval][Epoch 13][Iter: 0/16]CELoss: 4.67737, loss: 4.67737, top1: 0.00000, top5: 0.00000, batch_cost: 0.81981s, reader_cost: 0.44846, ips: 78.06683 images/sec +[2022/04/18 10:35:46] root INFO: [Eval][Epoch 13][Iter: 10/16]CELoss: 4.60637, loss: 4.60637, top1: 0.00000, top5: 0.06250, batch_cost: 0.53856s, reader_cost: 0.15151, ips: 118.83628 images/sec +[2022/04/18 10:35:49] root INFO: [Eval][Epoch 13][Avg]CELoss: 4.59937, loss: 4.59937, top1: 0.01146, top5: 0.05729 +[2022/04/18 10:35:49] root INFO: [Eval][Epoch 13][best metric: 0.015625] +[2022/04/18 10:35:49] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_13 +[2022/04/18 10:35:49] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:35:57] root INFO: [Train][Epoch 14/20][Iter: 0/4]lr: 0.00938, top1: 0.01953, top5: 0.08203, CELoss: 4.57910, loss: 4.57910, batch_cost: 6.31182s, reader_cost: 0.59390, ips: 40.55886 images/sec, eta: 0:02:56 +[2022/04/18 10:36:07] root INFO: [Train][Epoch 14/20][Avg]top1: 0.01172, top5: 0.07552, CELoss: 4.58491, loss: 4.58491 +[2022/04/18 10:36:07] root INFO: [Eval][Epoch 14][Iter: 0/16]CELoss: 4.67771, loss: 4.67771, top1: 0.00000, top5: 0.01562, batch_cost: 0.81435s, reader_cost: 0.45404, ips: 78.59002 images/sec +[2022/04/18 10:36:13] root INFO: [Eval][Epoch 14][Iter: 10/16]CELoss: 4.63535, loss: 4.63535, top1: 0.00000, top5: 0.04688, batch_cost: 0.55286s, reader_cost: 0.14929, ips: 115.76124 images/sec +[2022/04/18 10:36:15] root INFO: [Eval][Epoch 14][Avg]CELoss: 4.59307, loss: 4.59307, top1: 0.01250, top5: 0.05625 +[2022/04/18 10:36:15] root INFO: [Eval][Epoch 14][best metric: 0.015625] +[2022/04/18 10:36:15] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_14 +[2022/04/18 10:36:15] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:36:23] root INFO: [Train][Epoch 15/20][Iter: 0/4]lr: 0.00849, top1: 0.01953, top5: 0.10547, CELoss: 4.57969, loss: 4.57969, batch_cost: 6.29119s, reader_cost: 0.59107, ips: 40.69185 images/sec, eta: 0:02:30 +[2022/04/18 10:36:34] root INFO: [Train][Epoch 15/20][Avg]top1: 0.01693, top5: 0.08594, CELoss: 4.58005, loss: 4.58005 +[2022/04/18 10:36:34] root INFO: [Eval][Epoch 15][Iter: 0/16]CELoss: 4.68616, loss: 4.68616, top1: 0.00000, top5: 0.00000, batch_cost: 0.83016s, reader_cost: 0.45301, ips: 77.09354 images/sec +[2022/04/18 10:36:41] root INFO: [Eval][Epoch 15][Iter: 10/16]CELoss: 4.65745, loss: 4.65745, top1: 0.00000, top5: 0.06250, batch_cost: 0.66344s, reader_cost: 0.16827, ips: 96.46697 images/sec +[2022/04/18 10:36:43] root INFO: [Eval][Epoch 15][Avg]CELoss: 4.59027, loss: 4.59027, top1: 0.01146, top5: 0.05625 +[2022/04/18 10:36:43] root INFO: [Eval][Epoch 15][best metric: 0.015625] +[2022/04/18 10:36:43] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_15 +[2022/04/18 10:36:43] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:36:52] root INFO: [Train][Epoch 16/20][Iter: 0/4]lr: 0.00755, top1: 0.01562, top5: 0.07812, CELoss: 4.56924, loss: 4.56924, batch_cost: 6.29722s, reader_cost: 0.59081, ips: 40.65288 images/sec, eta: 0:02:05 +[2022/04/18 10:37:02] root INFO: [Train][Epoch 16/20][Avg]top1: 0.01432, top5: 0.06641, CELoss: 4.56814, loss: 4.56814 +[2022/04/18 10:37:03] root INFO: [Eval][Epoch 16][Iter: 0/16]CELoss: 4.68624, loss: 4.68624, top1: 0.00000, top5: 0.00000, batch_cost: 0.88268s, reader_cost: 0.45880, ips: 72.50629 images/sec +[2022/04/18 10:37:09] root INFO: [Eval][Epoch 16][Iter: 10/16]CELoss: 4.65055, loss: 4.65055, top1: 0.00000, top5: 0.04688, batch_cost: 0.56403s, reader_cost: 0.15235, ips: 113.46999 images/sec +[2022/04/18 10:37:11] root INFO: [Eval][Epoch 16][Avg]CELoss: 4.58911, loss: 4.58911, top1: 0.00937, top5: 0.05000 +[2022/04/18 10:37:11] root INFO: [Eval][Epoch 16][best metric: 0.015625] +[2022/04/18 10:37:11] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_16 +[2022/04/18 10:37:11] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:37:20] root INFO: [Train][Epoch 17/20][Iter: 0/4]lr: 0.00658, top1: 0.00781, top5: 0.08594, CELoss: 4.57454, loss: 4.57454, batch_cost: 6.29410s, reader_cost: 0.58915, ips: 40.67300 images/sec, eta: 0:01:40 +[2022/04/18 10:37:30] root INFO: [Train][Epoch 17/20][Avg]top1: 0.01172, top5: 0.07161, CELoss: 4.57665, loss: 4.57665 +[2022/04/18 10:37:31] root INFO: [Eval][Epoch 17][Iter: 0/16]CELoss: 4.67491, loss: 4.67491, top1: 0.00000, top5: 0.00000, batch_cost: 0.87708s, reader_cost: 0.45714, ips: 72.96934 images/sec +[2022/04/18 10:37:37] root INFO: [Eval][Epoch 17][Iter: 10/16]CELoss: 4.64649, loss: 4.64649, top1: 0.04688, top5: 0.07812, batch_cost: 0.54974s, reader_cost: 0.15248, ips: 116.41848 images/sec +[2022/04/18 10:37:39] root INFO: [Eval][Epoch 17][Avg]CELoss: 4.58624, loss: 4.58624, top1: 0.01354, top5: 0.06042 +[2022/04/18 10:37:39] root INFO: [Eval][Epoch 17][best metric: 0.015625] +[2022/04/18 10:37:39] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_17 +[2022/04/18 10:37:39] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:37:48] root INFO: [Train][Epoch 18/20][Iter: 0/4]lr: 0.00560, top1: 0.01953, top5: 0.07422, CELoss: 4.58180, loss: 4.58180, batch_cost: 6.30441s, reader_cost: 0.59464, ips: 40.60648 images/sec, eta: 0:01:15 +[2022/04/18 10:37:58] root INFO: [Train][Epoch 18/20][Avg]top1: 0.01172, top5: 0.08073, CELoss: 4.56663, loss: 4.56663 +[2022/04/18 10:37:59] root INFO: [Eval][Epoch 18][Iter: 0/16]CELoss: 4.67365, loss: 4.67365, top1: 0.00000, top5: 0.00000, batch_cost: 0.88373s, reader_cost: 0.47119, ips: 72.42037 images/sec +[2022/04/18 10:38:05] root INFO: [Eval][Epoch 18][Iter: 10/16]CELoss: 4.62118, loss: 4.62118, top1: 0.04688, top5: 0.09375, batch_cost: 0.55812s, reader_cost: 0.15676, ips: 114.67145 images/sec +[2022/04/18 10:38:07] root INFO: [Eval][Epoch 18][Avg]CELoss: 4.58644, loss: 4.58644, top1: 0.01250, top5: 0.06042 +[2022/04/18 10:38:07] root INFO: [Eval][Epoch 18][best metric: 0.015625] +[2022/04/18 10:38:07] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_18 +[2022/04/18 10:38:07] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:38:15] root INFO: [Train][Epoch 19/20][Iter: 0/4]lr: 0.00463, top1: 0.02344, top5: 0.07422, CELoss: 4.55407, loss: 4.55407, batch_cost: 6.29300s, reader_cost: 0.59351, ips: 40.68010 images/sec, eta: 0:00:50 +[2022/04/18 10:38:26] root INFO: [Train][Epoch 19/20][Avg]top1: 0.01432, top5: 0.07422, CELoss: 4.56636, loss: 4.56636 +[2022/04/18 10:38:26] root INFO: [Eval][Epoch 19][Iter: 0/16]CELoss: 4.67197, loss: 4.67197, top1: 0.00000, top5: 0.00000, batch_cost: 0.84170s, reader_cost: 0.45035, ips: 76.03652 images/sec +[2022/04/18 10:38:32] root INFO: [Eval][Epoch 19][Iter: 10/16]CELoss: 4.63389, loss: 4.63389, top1: 0.04688, top5: 0.06250, batch_cost: 0.55268s, reader_cost: 0.15200, ips: 115.80008 images/sec +[2022/04/18 10:38:35] root INFO: [Eval][Epoch 19][Avg]CELoss: 4.58614, loss: 4.58614, top1: 0.00937, top5: 0.06042 +[2022/04/18 10:38:35] root INFO: [Eval][Epoch 19][best metric: 0.015625] +[2022/04/18 10:38:35] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_19 +[2022/04/18 10:38:35] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest +[2022/04/18 10:38:44] root INFO: [Train][Epoch 20/20][Iter: 0/4]lr: 0.00371, top1: 0.00781, top5: 0.05469, CELoss: 4.57822, loss: 4.57822, batch_cost: 6.30198s, reader_cost: 0.59387, ips: 40.62213 images/sec, eta: 0:00:25 +[2022/04/18 10:38:55] root INFO: [Train][Epoch 20/20][Avg]top1: 0.01823, top5: 0.07422, CELoss: 4.56660, loss: 4.56660 +[2022/04/18 10:38:56] root INFO: [Eval][Epoch 20][Iter: 0/16]CELoss: 4.68744, loss: 4.68744, top1: 0.00000, top5: 0.00000, batch_cost: 1.02800s, reader_cost: 0.47287, ips: 62.25709 images/sec +[2022/04/18 10:39:02] root INFO: [Eval][Epoch 20][Iter: 10/16]CELoss: 4.63357, loss: 4.63357, top1: 0.00000, top5: 0.09375, batch_cost: 0.63272s, reader_cost: 0.16321, ips: 101.15126 images/sec +[2022/04/18 10:39:05] root INFO: [Eval][Epoch 20][Avg]CELoss: 4.58696, loss: 4.58696, top1: 0.00833, top5: 0.06563 +[2022/04/18 10:39:05] root INFO: [Eval][Epoch 20][best metric: 0.015625] +[2022/04/18 10:39:05] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\epoch_20 +[2022/04/18 10:39:05] root INFO: Already save model in ./output/ShuffleNetV2_x0_25\latest