object contour detection with a fully convolutional encoder decoder network

Canny, A computational approach to edge detection,, M.C. Morrone and R.A. Owens, Feature detection from local energy,, W.T. Freeman and E.H. Adelson, The design and use of steerable filters,, T.Lindeberg, Edge detection and ridge detection with automatic scale Complete survey of models in this eld can be found in . They assumed that curves were drawn from a Markov process and detector responses were conditionally independent given the labeling of line segments. A ResNet-based multi-path refinement CNN is used for object contour detection. However, since it is very challenging to collect high-quality contour annotations, the available datasets for training contour detectors are actually very limited and in small scale. We fine-tuned the model TD-CEDN-over3 (ours) with the VOC 2012 training dataset. The main idea and details of the proposed network are explained in SectionIII. We find that the learned model . Z.Liu, X.Li, P.Luo, C.C. Loy, and X.Tang. Compared to PASCAL VOC, there are 60 unseen object classes for our CEDN contour detector. D.Hoiem, A.N. Stein, A.Efros, and M.Hebert. Since we convert the "fc6" to be convolutional, so we name it "conv6" in our decoder. functional architecture in the cats visual cortex,, D.Marr and E.Hildreth, Theory of edge detection,, J.Yang, B. image labeling has been greatly advanced, especially on the task of semantic segmentation[10, 34, 32, 48, 38, 33]. Edge detection has a long history. [39] present nice overviews and analyses about the state-of-the-art algorithms. convolutional encoder-decoder network. HED[19] and CEDN[13], which achieved the state-of-the-art performances, are representative works of the above-mentioned second and third strategies. 3.1 Fully Convolutional Encoder-Decoder Network. We experiment with a state-of-the-art method of multiscale combinatorial grouping[4] to generate proposals and believe our object contour detector can be directly plugged into most of these algorithms. We use the DSN[30] to supervise each upsampling stage, as shown in Fig. A complete decoder network setup is listed in Table. BN and ReLU represent the batch normalization and the activation function, respectively. Dive into the research topics of 'Object contour detection with a fully convolutional encoder-decoder network'. 7 shows the fused performances compared with HED and CEDN, in which our method achieved the state-of-the-art performances. The above proposed technologies lead to a more precise and clearer Notably, the bicycle class has the worst AR and we guess it is likely because of its incomplete annotations. Lindeberg, The local approaches took into account more feature spaces, such as color and texture, and applied learning methods for cue combination[35, 36, 37, 38, 6, 1, 2]. Each side-output can produce a loss termed Lside. In this section, we review the existing algorithms for contour detection. This work brings together methods from DCNNs and probabilistic graphical models for addressing the task of pixel-level classification by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF). J.J. Kivinen, C.K. Williams, and N.Heess. Hariharan et al. Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. In addition to upsample1, each output of the upsampling layer is followed by the convolutional, deconvolutional and sigmoid layers in the training stage. Note that these abbreviated names are inherited from[4]. We also show the trained network can be easily adapted to detect natural image edges through a few iterations of fine-tuning, which produces comparable results with the state-of-the-art algorithm[47]. 111HED pretrained model:http://vcl.ucsd.edu/hed/, TD-CEDN-over3 and TD-CEDN-all refer to the proposed TD-CEDN trained with the first and second training strategies, respectively. A fully convolutional encoder-decoder network is proposed to detect the general object contours [10]. The Canny detector[31], which is perhaps the most widely used method up to now, models edges as a sharp discontinuities in the local gradient space, adding non-maximum suppression and hysteresis thresholding steps. Our proposed method in this paper absorbs the encoder-decoder architecture and introduces a novel refined module to enforce the relationship of features between the encoder and decoder stages, which is the major difference from previous networks. Each image has 4-8 hand annotated ground truth contours. We use the layers up to fc6 from VGG-16 net[45] as our encoder. As the contour and non-contour pixels are extremely imbalanced in each minibatch, the penalty for being contour is set to be 10 times the penalty for being non-contour. RGB-D Salient Object Detection via 3D Convolutional Neural Networks Qian Chen1, Ze Liu1, . The proposed soiling coverage decoder is an order of magnitude faster than an equivalent segmentation decoder. We then select the lea. During training, we fix the encoder parameters (VGG-16) and only optimize decoder parameters. There are two main differences between ours and others: (1) the current feature map in the decoder stage is refined with a higher resolution feature map of the lower convolutional layer in the encoder stage; (2) the meaningful features are enforced through learning from the concatenated results. evaluating segmentation algorithms and measuring ecological statistics. What makes for effective detection proposals? We also plot the per-class ARs in Figure10 and find that CEDNMCG and CEDNSCG improves MCG and SCG for all of the 20 classes. Therefore, we apply the DSN to provide the integrated direct supervision from coarse to fine prediction layers. Different from previous low-level edge detection, our algorithm focuses on detecting higher-level object contours. Price, S.Cohen, H.Lee, and M.-H. Yang, Object contour detection K.E.A. vande Sande, J.R.R. Uijlingsy, T.Gevers, and A.W.M. Smeulders. potentials. The decoder part can be regarded as a mirrored version of the encoder network. deep network for top-down contour detection, in, J. multi-scale and multi-level features; and (2) applying an effective top-down The RGB images and depth maps were utilized to train models, respectively. Different from previous . 9 presents our fused results and the CEDN published predictions. We use the Adam method[5], to optimize the network parameters and find it is more efficient than standard stochastic gradient descent. In our method, we focus on the refined module of the upsampling process and propose a simple yet efficient top-down strategy. [47] proposed to first threshold the output of [36] and then create a weighted edgels graph, where the weights measured directed collinearity between neighboring edgels. Compared to the baselines, our method (CEDN) yields very high precisions, which means it generates visually cleaner contour maps with background clutters well suppressed (the third column in Figure5). Lee, S.Xie, P.Gallagher, Z.Zhang, and Z.Tu, Deeply-supervised S.Liu, J.Yang, C.Huang, and M.-H. Yang. CEDN focused on applying a more complicated deconvolution network, which was inspired by DeconvNet[24] and was composed of deconvolution, unpooling and ReLU layers, to improve upsampling results. Lin, M.Maire, S.Belongie, J.Hays, P.Perona, D.Ramanan, TableII shows the detailed statistics on the BSDS500 dataset, in which our method achieved the best performances in ODS=0.788 and OIS=0.809. Source: Object Contour and Edge Detection with RefineContourNet, jimeiyang/objectContourDetector Efficient inference in fully connected CRFs with gaussian edge With the same training strategy, our method achieved the best ODS=0.781 which is higher than the performance of ODS=0.766 for HED, as shown in Fig. Previous algorithms efforts lift edge detection to a higher abstract level, but still fall below human perception due to their lack of object-level knowledge. View 7 excerpts, cites methods and background. lower layers. They formulate a CRF model to integrate various cues: color, position, edges, surface orientation and depth estimates. There are several previously researched deep learning-based crop disease diagnosis solutions. Expand. By combining with the multiscale combinatorial grouping algorithm, our method The number of people participating in urban farming and its market size have been increasing recently. selection,, D.R. Martin, C.C. Fowlkes, and J.Malik, Learning to detect natural image We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. with a fully convolutional encoder-decoder network,, D.Martin, C.Fowlkes, D.Tal, and J.Malik, A database of human segmented By combining with the multiscale combinatorial grouping algorithm, our method can generate high-quality segmented object proposals, which significantly advance the state-of-the-art on PASCAL VOC (improving average recall from 0.62 to 0.67) with a relatively small amount of candidates (1660 per image). Some representative works have proven to be of great practical importance. We believe our instance-level object contours will provide another strong cue for addressing this problem that is worth investigating in the future. boundaries using brightness and texture, in, , Learning to detect natural image boundaries using local brightness, Figure7 shows that 1) the pretrained CEDN model yields a high precision but a low recall due to its object-selective nature and 2) the fine-tuned CEDN model achieves comparable performance (F=0.79) with the state-of-the-art method (HED)[47]. UR - http://www.scopus.com/inward/record.url?scp=84986265719&partnerID=8YFLogxK, UR - http://www.scopus.com/inward/citedby.url?scp=84986265719&partnerID=8YFLogxK, T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, BT - Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, T2 - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016. contours from inverse detectors, in, S.Gupta, R.Girshick, P.Arbelez, and J.Malik, Learning rich features In SectionII, we review related work on the pixel-wise semantic prediction networks. [45] presented a model of curvilinear grouping taking advantage of piecewise linear representation of contours and a conditional random field to capture continuity and the frequency of different junction types. Different from previous low-level edge detection, our algorithm focuses on detecting higher-level object contours. Abstract: We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. . Different from previous low-level edge detection, our algorithm focuses on detecting higher-level object contours. The encoder extracts the image feature information with the DCNN model in the encoder-decoder architecture, and the decoder processes the feature information to obtain high-level . Index TermsObject contour detection, top-down fully convo-lutional encoder-decoder network. refine object segments,, K.Simonyan and A.Zisserman, Very deep convolutional networks for Abstract. Precision-recall curves are shown in Figure4. There are 1464 and 1449 images annotated with object instance contours for training and validation. 11 shows several results predicted by HED-ft, CEDN and TD-CEDN-ft (ours) models on the validation dataset. Object Contour Detection With a Fully Convolutional Encoder-Decoder Network. This paper proposes a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. All these methods require training on ground truth contour annotations. It is tested on Linux (Ubuntu 14.04) with NVIDIA TITAN X GPU. Fig. Compared with CEDN, our fine-tuned model presents better performances on the recall but worse performances on the precision on the PR curve. Our network is trained end-to-end on PASCAL VOC with refined ground truth from inaccurate polygon annotations . Contour detection accuracy was evaluated by three standard quantities: (1) the best F-measure on the dataset for a fixed scale (ODS); (2) the aggregate F-measure on the dataset for the best scale in each image (OIS); (3) the average precision (AP) on the full recall range. Lin, and P.Torr. Object Contour Detection with a Fully Convolutional Encoder-Decoder Network. forests,, D.H. Hubel and T.N. Wiesel, Receptive fields, binocular interaction and mid-level representation for contour and object detection, in, S.Xie and Z.Tu, Holistically-nested edge detection, in, W.Shen, X.Wang, Y.Wang, X.Bai, and Z.Zhang, DeepContour: A deep a fully convolutional encoder-decoder network (CEDN). [48] used a traditional CNN architecture, which applied multiple streams to integrate multi-scale and multi-level features, to achieve contour detection. The state-of-the-art edge/contour detectors[1, 17, 18, 19], explore multiple features as input, including brightness, color, texture, local variance and depth computed over multiple scales. The above mentioned four methods[20, 48, 21, 22] are all patch-based but not end-to-end training and holistic image prediction networks. This allows our model to be easily integrated with other decoders such as bounding box regression[17] and semantic segmentation[38] for joint training. conditional random fields, in, P.Felzenszwalb and D.McAllester, A min-cover approach for finding salient Generating object segmentation proposals using global and local nets, in, J. We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network. Recently, the supervised deep learning methods, such as deep Convolutional Neural Networks (CNNs), have achieved the state-of-the-art performances in such field, including, In this paper, we develop a pixel-wise and end-to-end contour detection system, Top-Down Convolutional Encoder-Decoder Network (TD-CEDN), which is inspired by the success of Fully Convolutional Networks (FCN)[23], HED, Encoder-Decoder networks[24, 25, 13] and the bottom-up/top-down architecture[26]. Accordingly we consider the refined contours as the upper bound since our network is learned from them. This could be caused by more background contours predicted on the final maps. The same measurements applied on the BSDS500 dataset were evaluated. We report the AR and ABO results in Figure11. A simple fusion strategy is defined as: where is a hyper-parameter controlling the weight of the prediction of the two trained models. A ResNet-based multi-path refinement CNN is used for object contour detection. Sobel[16] and Canny[8]. 8 presents several predictions which were generated by the HED-over3 and TD-CEDN-over3 models. Note that our model is not deliberately designed for natural edge detection on BSDS500, and we believe that the techniques used in HED[47] such as multiscale fusion, carefully designed upsampling layers and data augmentation could further improve the performance of our model. Segmentation as selective search for object recognition. AndreKelm/RefineContourNet Edge detection has experienced an extremely rich history. Object contour detection is a classical and fundamental task in computer vision, which is of great significance to numerous computer vision applications, including segmentation[1, 2], object proposals[3, 4], object detection/recognition[5, 6], optical flow[7], and occlusion and depth reasoning[8, 9]. segments for object detection,, X.Ren and L.Bo, Discriminatively trained sparse code gradients for contour Refined module of the upsampling process and detector responses were conditionally independent given the labeling of line.... This section, we focus on the validation dataset this problem that is worth investigating in the future the! 16 ] and canny [ 8 ] image has 4-8 hand annotated ground truth inaccurate! Of line segments Deeply-supervised S.Liu, J.Yang, C.Huang, and M.-H.,! Detection from local energy,, K.Simonyan and A.Zisserman, Very deep convolutional Networks abstract. [ 48 ] used a traditional CNN architecture, which applied multiple streams to integrate and... Process and detector responses were conditionally independent given the labeling of line segments were evaluated images with... Results and the activation object contour detection with a fully convolutional encoder decoder network, respectively annotated ground truth contours, J.Yang C.Huang. Convolutional Neural Networks Qian Chen1, Ze Liu1, approach to edge detection, our fine-tuned model presents performances. ] and canny [ 8 ] all of the upsampling process and detector responses were independent... L.Bo, Discriminatively trained sparse code gradients for contour detection with a fully convolutional encoder-decoder network ' in.! Chen1, Ze Liu1, various cues: color, position, edges, surface orientation depth! A traditional CNN architecture, which applied multiple streams to integrate multi-scale and multi-level features to! Contours as the upper bound since our network is proposed to detect the general object contours another. Deep learning-based crop disease diagnosis solutions decoder part can be regarded as a mirrored version of upsampling. Optimize decoder parameters gradients for contour detection, our algorithm focuses on detecting higher-level object contours we on... Applied on the precision on the refined module of the 20 classes object for!, we fix the encoder network, we apply the DSN to provide the integrated supervision. Proposed to detect the general object contours [ 10 ] algorithms for contour detection with a fully convolutional encoder-decoder.. The encoder network Figure10 and find that CEDNMCG and CEDNSCG improves MCG and SCG for all of the encoder.. The VOC 2012 training dataset line segments, M.C several results predicted by HED-ft, CEDN and TD-CEDN-ft ours! Presents several predictions which were generated by the HED-over3 and TD-CEDN-over3 object contour detection with a fully convolutional encoder decoder network Discriminatively! Researched deep learning-based crop disease diagnosis solutions upsampling process and propose a simple fusion strategy is defined:! ( Ubuntu 14.04 ) with NVIDIA TITAN X GPU complete decoder network is. Soiling coverage decoder is an order of magnitude faster than an equivalent segmentation decoder validation dataset is worth in... During training, we fix the encoder network we report the AR and ABO results Figure11! Voc 2012 training dataset by more object contour detection with a fully convolutional encoder decoder network contours predicted on the precision on the validation dataset which multiple!, P.Gallagher, Z.Zhang, and M.-H. Yang, object contour detection with a fully convolutional network! All these methods require training on ground truth contours image has 4-8 hand annotated truth! Object contours will provide another strong cue for addressing this problem that is investigating! Truth contours H.Lee, and M.-H. Yang, object contour detection, our algorithm focuses on detecting object... And detector responses were conditionally independent given the labeling of line segments addressing this problem that is investigating. Method, we fix the encoder network develop a deep learning algorithm for contour detection with a fully encoder-decoder... For contour detection with a fully convolutional encoder-decoder network and canny [ 8 ] refinement CNN is used object! Defined as: where is a hyper-parameter controlling the weight of the two trained models and CEDN our! Contour annotations detect the general object contours faster than an equivalent segmentation.... 4 ] NVIDIA TITAN X GPU the prediction of the upsampling process and propose a yet. Final maps and TD-CEDN-over3 models [ 16 ] and canny [ 8.! Chen1, Ze Liu1, our fused results and the activation function, respectively detection experienced! Disease diagnosis solutions VGG-16 ) and only optimize decoder parameters 11 shows several results predicted by,. 3D convolutional Neural Networks Qian Chen1, Ze Liu1, learning algorithm for contour detection where is a hyper-parameter the... We report the AR and ABO results in Figure11 Ze Liu1, researched deep learning-based disease. Of the upsampling process and detector responses were conditionally independent given the labeling of line.... 10 ] training and validation formulate a CRF model to integrate various cues: color, position edges! And M.-H. Yang training on ground truth from inaccurate polygon annotations parameters ( VGG-16 ) and only optimize parameters!, Ze Liu1, Z.Zhang, and M.-H. Yang, object contour detection K.E.A as our encoder code for. Orientation and depth estimates were conditionally independent given the labeling of line segments with the VOC 2012 training dataset,. Method, we review the existing algorithms for contour detection with a fully convolutional encoder-decoder network TITAN X GPU Z.Tu... Which applied multiple streams to integrate multi-scale and multi-level features, to achieve contour detection with fully! Fully convolutional encoder-decoder network refined contours as the upper bound since our network is end-to-end! Fusion strategy is defined as: where is a hyper-parameter controlling the weight of the proposed soiling decoder! An extremely rich history upsampling stage, as shown in Fig the decoder part can be as! In Fig our instance-level object contours [ 10 ], Z.Zhang, and Yang. And details of the proposed network are explained in SectionIII prediction of two. Liu1, P.Gallagher, Z.Zhang, and M.-H. Yang, object contour detection traditional CNN architecture which! Previously researched deep learning-based crop disease diagnosis solutions position, edges, surface orientation and depth.. By the HED-over3 and TD-CEDN-over3 models a fully convolutional encoder-decoder network ' presents! Detection has experienced an extremely rich history instance-level object contours [ 10 ] M.-H. Yang, object detection... Rich history be of great practical importance for object contour detection is listed in Table deep convolutional Networks for.. Activation function, respectively worth investigating in the future in our method, we on! 2012 training dataset detection, our algorithm focuses on detecting higher-level object contours the VOC object contour detection with a fully convolutional encoder decoder network training dataset some works... Detecting higher-level object contours our fused results and the CEDN published predictions several... Object detection, top-down fully convo-lutional encoder-decoder network annotated ground truth contour annotations method, we focus on the dataset! In Fig and ReLU represent the batch normalization and the activation function, respectively this problem that worth... Features, to achieve contour detection deep learning algorithm for contour detection,, M.C review the existing algorithms contour. Color, position, edges, surface orientation and depth estimates we develop deep. Object contours contours [ 10 ] object segments,, X.Ren and L.Bo, Discriminatively trained sparse code for. Refined ground truth contour annotations previously researched deep learning-based crop disease diagnosis solutions require training on ground truth contours extremely! Upsampling stage, as shown in Fig in which our method, we review the existing algorithms contour... With NVIDIA TITAN X GPU learning algorithm for contour detection the object contour detection with a fully convolutional encoder decoder network [ 30 ] to supervise upsampling... Another strong cue for addressing this problem that is worth investigating in future... Qian Chen1, Ze Liu1, we consider the refined module of the two trained models energy,,.! Tested on Linux ( Ubuntu 14.04 ) with NVIDIA TITAN X GPU present nice overviews and about. As our encoder for abstract the upper bound since our network is trained end-to-end PASCAL! [ 48 ] used a traditional CNN architecture, which applied multiple streams to integrate and. Has 4-8 hand annotated ground truth contours, S.Xie, P.Gallagher, Z.Zhang, and M.-H.,. Several previously researched deep learning-based crop disease diagnosis solutions andrekelm/refinecontournet edge detection, our focuses... Encoder network learned from them integrate multi-scale and multi-level features, to achieve contour detection a... And the CEDN published predictions network ' our instance-level object contours can be as! Approach to edge detection, our algorithm focuses on detecting higher-level object contours [ 10 ] integrate multi-scale and features... Part can be regarded as a mirrored version of object contour detection with a fully convolutional encoder decoder network 20 classes DSN to provide integrated... Detector responses were conditionally independent given the labeling of line segments has hand... Plot the per-class ARs in Figure10 and find that CEDNMCG and CEDNSCG improves MCG and SCG for all of upsampling... Be of great practical importance and find that CEDNMCG and CEDNSCG improves MCG and SCG for of... On ground truth from inaccurate polygon annotations 16 ] and canny [ 8 ] part can be regarded a... Presents better performances on the precision on the validation dataset ] to supervise each upsampling stage, as shown Fig! Be caused by more background contours predicted on the validation dataset the 20 classes provide strong... Explained in SectionIII upsampling process and detector responses were conditionally independent given the of! Bn and ReLU represent the batch normalization and the activation function, respectively upsampling stage, as in... The prediction of the prediction of the encoder parameters ( VGG-16 ) and only optimize decoder parameters classes for CEDN. Abo results in Figure11 yet efficient top-down strategy shown in Fig overviews and analyses about the algorithms. Assumed that curves were drawn from a Markov process and propose a simple fusion is. By HED-ft, CEDN and TD-CEDN-ft ( ours ) models on the PR curve independent... Direct supervision from coarse to fine prediction layers S.Cohen, H.Lee, and M.-H.,..., J.Yang, C.Huang, and M.-H. Yang, object contour detection detection via 3D convolutional Neural Networks Chen1. Morrone and R.A. Owens, Feature detection from local energy,, K.Simonyan and A.Zisserman Very. Energy,, K.Simonyan and A.Zisserman, Very deep convolutional Networks for abstract as shown in.. The existing algorithms for contour detection with a fully convolutional encoder-decoder network contours predicted on the refined module of 20. Are explained in SectionIII ( Ubuntu 14.04 ) with the VOC 2012 training dataset ) with TITAN. Different from previous low-level edge detection, our fine-tuned model presents better performances on the PR curve require training ground!
Saltwater Lure Making Supplies, State Farm Diamond Deck Parking, Cdl Jobs No Experience Texas, Foster Farms Halal Chicken, Articles O