2017 Fifth International Conference on Information and Communication Technology (ICoICT)
An Implementation of Convolutional Neural Network
on PCO Classification based on Ultrasound Image
B. Cahyono1, Adiwijaya2, M. S. Mubarok3, U.N. Wisesty4
School of Computing
Telkom University
Bandung 40257, Indonesia
1
baguscah77@[Link], 2adiwijaya@[Link], 3msyahrulmubarok@[Link],
4
untarinw@[Link]
Abstract— Polycystic ovary syndrome (PCOS) is a hormonal image. In this research, the feature extraction is done
endocrine disorder that infect many women in their reproductive automatically by system. So, we hope it will ensure the
cycle. It is a concern in a married couple because it is related robustness of the system to handle real world data.
fertility rate of women. One of the criteria for diagnosing PCOS
are polycystic ovaries (PCO). Polycystic ovaries can be seen from II. RELATED WORKS
the number and diameter of each follicle on ultrasound image. In
Adiwijaya et al. in [6] use region growing method and
previous studies, there are existing PCO classifications done
Euclidian distance to detect follicle and to measure follicle size
automatically by the system using several methods. However, on
those studies its feature extraction of the ultrasound image is still with accuracy of 78%. Here, there is no classification, only
done manually. In this research, we propose a solution where the follicle detection and measurement.
feature extraction is also done automatically using Convolutional Seatiawati et al. in [7] use Particle Swarm Optimization to
Neural Network. CNN provide the best test performance with cluster ultrasound image with modified non-parametric fitness
micro-average f1-score of 100% and an average of 76.36% on a 5- function. Using a contrast enhancement can produce better result
fold cross-validation.
on follicle measurement. Machine Learning approaches are
suggested to classify all follicle features that already extracted.
Keywords— Policystic Ovary Syndrome; polycystic ovaries;
ultrasound images; Convolutional Neural Network; Purnama et al. in [8] use three machine learning approach to
classify ultrasound image that is NN-LVQ, KNN using
I. INTRODUCTION Euclidian distance, and SVM. Ultrasound image preprocessed
Polycystic Ovary Syndrome (PCOS), also called using low pass filter, histogram average, binarization,
Hyperadrogenic Syndrome, is hormonal endocrine disorders morphology to obtain binary image, and edge detection. The
that infect many women in their reproductive cycle. It is a feature of the image still extracted manually using Gabor
concern in a married couple because it is related to a fertility rate Wavelet before going to classification. The best result is 82.55%
of women [1]. PCOS are affecting 5-10% women on age 18-44 using SVM and 78.81% using KNN with k=5.
years [2, 3]. Due to the criteria to determine PCOS are still
changing there is no exact value how many women are affected Meanwhile, Wisesty [Link] in [9] provides the modification of
but estimated one of 10-15 woman are affected with PCOS. backpropagation algorithm for Polycystic Ovary Syndrome
Many women are diagnosed of having PCOS when they reach Detection Based on Ultrasound Images. The backpropagation is
age 20-30. However, PCOS can also affecting woman as young modified by using Marquardt optimization and Conjugate
as 11 years old [4]. Gradient - Fletcher Reeves to improve the convergence rate.
In the 2012 NIH Conference confirm that the Rotterdam III. PROPOSED SCHEME
criteria is the best criteria to detect PCOS, and Endocrine Society Our proposed system is using a deep learning approach to
Guideline also supporting these criteria [5]. Rotterdam criteria classify ultrasound images. Specifically, we are using
are criteria that proposed in 2003, One of the Rotterdam criteria Convolutional Neural Network that the best approach for image
is Polycystic Ovaries (PCO) [1]. The criteria to detect PCO that classification [10]. Feature extraction is done automatically
is at least one of the ovary have volume equal or more than 10 using deep learning so we are not adding any feature extraction
cm3, or have more than 12 follicles with 2-9 mm diameter [6]. process to the data before going into the system.
But still the doctor need to analyze the ultrasound image to
indicate PCO. The dataset we use had been labeled by specialist.
Dataset are divided into 3 categories, training data by 60%,
Using Convolutional Neural Network (CNN) we try to validation data by 20%, and testing data by 20%. Our dataset is
classify the ultrasound image to determine PCO class or non- composed with total 40 non-PCO data and 14 PCO data. Train
PCO class. CNN used because its already proved to be the best data gets 24 non-PCO data and 8 PCO data, validation and
approach for image classification [10]. In the related works testing each gets 8 non-PCO data and 3 PCO data. Since our train
section, there are some system that classify ultrasound image data is not balanced at all, we duplicate PCO data as if the data
automatically using many approaches. But all of them still use becomes balanced. Duplication really help when training the
manual extraction method to extract feature of the ultrasound network, because if train data stay unbalanced it will make the
Authorized licensed use limited to: J.R.D. Tata Memorial Library Indian Institute of Science Bengaluru. Downloaded on April 06,2025 at [Link] UTC from IEEE Xplore. Restrictions apply.
ISBN: 978-1-5090-4911-0 (c) 2017 IEEE
network more likely to lead non-PCO data and cannot recognize 4x6 to 2x3 so the remaining layers are not affected with the
PCO data at all [11]. change of data dimension.
We use accuracy to measure network performance each
epoch to determine which epoch are the best. But in testing
phase, we use F1-Measure and Micro-average F1-Measure to
measure the performance of the network. F1-Measure is used
because it has both Precision and Recall measurement. F1-
Measure also measure performance of each class, so we know
how the network behave with PCO data and non PCO data.
Micro-average F1-Measure is used to sums up the F1-Measure
Fig. 1. Illustraion of the Proposed Architecture. performance results from both PCO data and non PCO data.
In Fig. 1, we see the general view of the proposed system. IV. RESULT AND DISCUSSION
The proposed architecture consists of 6 layers with 2 In this first observation, we try to find dropout rate and
convolution layer 2 poling layers and 2 fully connected layers. learning rate that make the network converge faster and still
The proposed architecture may look simple, but it is quite maintain to produce the best result. In this observation, we try 6
powerful and easy to train. different combination of dropout rate and learning rate. The
In Table 1 we describe in detail entire architecture with input, dropout rates value we observe are start from 0 or No Dropout
layer parameter, and output. In layer 2 (Pooling layer), we at all, 0.1, and 0.2. The learning rate value we observe is 10-6 and
reduce data dimensions and change it from 2:3 aspect ratio to 10-7. We try bigger learning rate but the softmax start breaking
1:1 aspect ratio (Square) to simplify calculation in the next and producing NaN in the first 10 epochs.
layers. In layer 4 the input data reshaped from 3D matrix to
single vector to make calculation faster. Softmax is used for
scoring function though dataset consist only two labels i.e. PCO
and non-PCO. Loss function is calculated using cross-entropy
loss.
TABLE I. DETAIL OF THE PROPOSED ARCHITECTURE
Data Dimension
Layer Type Parameters
Input Output
Convolution Layer 200x300x1 200x300x2 2 filters 7x7, pad 3
Pooling Layer 200x300x2 50x50x2 4x6 filter
Convolution Layer 50x50x2 50x50x4 4 filters 5x5, pad 2
Pooling Layer 50x50x4 25x25x4 2x2 filter
FC Layer 2500 64 -
FC Layer (Scooring) 64 2 - Fig. 2. Train and Validation Accuracies on Each Learning Rate and Dropout
Rate Combinations.
Dropout is used to prevent overfitting in the network, type of Fig. 2 shows training accuracy and validation accuracy for
dropout that is used is inverted dropout [12]. So rather than each learning rate and dropout rate combination. Combination
averaging the output value of each layer in testing and validation that use learning rate 10-6 and dropout rate 0.1 have the best train
phase, it amplifies other outputs that not dropped. This makes accuracy and validation accuracy. After choosing the best epoch
the calculation process in testing phase remains unchanged. The for each combination we run the testing process using the weight
dropout is implemented in each layer except the scoring layer. and bias in selected epoch.
Synaptic weights and convolution filters are initialized
randomly using normal distribution with mean 0 and variance
0.1. We use SGD method with momentum for gradient descent
and gradient optimization.
At first, we encounter many problems. Mostly are because
of the softmax activation is broken and the result becomes Not a
Number (NAN). But after the learning rate and dropout rate are
optimized the network run as expected. So, in this research we
show our observation about learning rate value and dropout rate
used. We also observe the performance of the network using
smaller data dimension. Smaller data dimension is sure
containing smaller information, but it makes the network
calculation faster. In this observation, we decrease the image
size from 200x300 to 100x150. The network architecture is
changed only in layer 2. Pooling filter in layer 2 is changed from Fig. 3. Test Performances on Each Learning Rate and Dropout Rate
Combinations.
Authorized licensed use limited to: J.R.D. Tata Memorial Library Indian Institute of Science Bengaluru. Downloaded on April 06,2025 at [Link] UTC from IEEE Xplore. Restrictions apply.
From Fig. 4 we see that 100x150 data dimension makes train We visualize what happens when the ultrasound image goes
and validation accuracies are low that is 53% and 82%. Even into the network as step by step process. The proposed
though the accuracies are low, with small data dimension the architecture is visualized using learning rate 10-6, rate dropout
learning time are faster rather than learning time the higher data 0.1, original data dimension, on fold 4 of 5-fold cross validation.
dimension.
The learning time for 100x150 data dimension is 26s per
epoch. The learning time for 200x300 data dimension is 55s per
epoch. The accuracies on smaller data dimension is low because
using smaller data dimension also means the network gets fewer
information, with fewer information the network get, it also
saves fewer information about the data.
Fig. 6. Visualization of (a) First Filter and (b) Second Filter that are Used for
Convolution in Layer 1.
In Fig. 5 we display the original image used as input to the
proposed architecture. In the first convolution layer, we have 2
filters with size 7x7. Also in Fig. 6 we display the filter used for
convolution in the first layer. The result of this convolution is
shown in Fig. 7.
Fig. 4. Performances on Each Data Dimension
From all observations, the best testing performance are
relatively low. We suspect it happens since the distribution in
dataset is not good. So, we run 5-fold cross-validation using the
best hyper parameters. From Table 2, we see that this
architecture capable making 100% micro-average f1-score in
testing phase with average of 76.36%. It proves that even only
with 6 layer, the network can classify USG image to PCO and
non PCO class is well. It also proves that the data is not well
balanced and too small. With much more dataset and well
balanced in both class we believe that this network can produce
much better performance.
V. NETWORK VISUALIZATION
(a)
Fig. 5. Data that is Used for This Network Visualization.
(b)
Fig. 7. (a) The Interest Area of First Filter (left) and Second Filter (right) for
Layer 1 and (b) The Interest Area After Pooling.
Authorized licensed use limited to: J.R.D. Tata Memorial Library Indian Institute of Science Bengaluru. Downloaded on April 06,2025 at [Link] UTC from IEEE Xplore. Restrictions apply.
In Fig. 7 section (a) we display the interest area of Layer 1. VI. CONCLUSION
Because we have 2 filters in Layer 1, so we display each filter Classification of ultrasound image using the Convolutional
independently. The brighter the area means it is more interested Neural Network to PCO and non PCO class has a good result
in that area. We hardly see the interest area in the result of and robust, since the system extract the feature of each image
convoluting with the filters in layer 1. But in Fig. 7 section (b), automatically. The proposed scheme does not need to explicitly
after the pooling layer we can see clearly the interest area of the define the method for feature extraction. The system can achieve
left filter. Based on Fig. 7 section (b) we know that both filter performance of 100% micro-average f1-score with average of
interest in dark area even though the output not identical but 76.36% in the testing phase. The proposed architecture can
somehow similar. improve the performance. Hyper parameters give the best
performance based on the proposed architecture are 10 learning 6
rate, 0.1 dropout rate, and original data dimension. In this
research, we also learn that optimizing CNN is hard, since many
times the softmax activation become NaN because of poor
weight initialization and too big learning rate. For future work,
we recommend using more dropout rate and better weight
initialization. We also recommend using dropout only in fully
connected layer, since in this experiment we apply dropout in all
Fig. 8. The Filters Used for Convolution in Layer 3
layers except scoring layer and the network breaks with higher
dropout rate.
In Fig. 8 we display filter used for convolution in layer 3.
Because each filter has 2 depths, we display each of depth REFERENCES
separately. First depth of filters is displayed at the top of Fig. 8
[1] Trivax, Bradley, and Ricardo Azziz. "Diagnosis of polycystic ovary
and second depth of filter are displayed at the bottom of Fig. 8. syndrome." Clinical obstetrics and gynecology 50, no. 1 (2007): 168-177.
The result of convolution using filters in Fig. 8 are shown in Fig. [2] NICHD, NIH. "DHHS." Beyond infertility: polycystic ovary syndrome
9. Again, in Fig. 9 we show the interest area of Layer 3. Identical (PCOS)(08-5863), Government Printing Office, Wshington, DC:
with filters in Layer 1, filters in Layer 3 are also divided into two US (2008): 791-798.
groups. We can see all filters have good output, not a single [3] National Center for Biotechnology Information, National Library of
output produce fully black image. This indicate that in this Medicine, PubMed Health. 2011. Polycystic ovary syndrome. Retrieved
network there is no dead ReLU problem appears. Dead ReLU February 8, 2016, from
[Link]
usually appears in network that use ReLU as activation function.
[4] Legro, Richard S., Silva A. Arslanian, David A. Ehrmann, Kathleen M.
Dead ReLU occurs when we use too big learning rate that makes Hoeger, M. Hassan Murad, Renato Pasquali, and Corrine K. Welt.
the neurons weights updates in such way that will makes all "Diagnosis and treatment of polycystic ovary syndrome: an Endocrine
outputs zero. Since the output is zero the neuron will never be Society clinical practice guideline." The Journal of Clinical
updated again in back propagation process. In this research, we Endocrinology & Metabolism 98, no. 12 (2013): 4565-4592.
carefully choose the learning rate so it can make the proposed [5] ESHRE, The Rotterdam, and ASRM-Sponsored PCOS Consensus
architecture avoided the dead ReLU problem. Workshop Group. "Revised 2003 consensus on diagnostic criteria and
long-term health risks related to polycystic ovary syndrome." Fertility and
sterility81, no. 1 (2004): 19-25.
[6] Purnama, B., A. Hasyim, M. D. Septiani, U. N. Wisesty, and W. Astuti.
"Follicle Detection on the USG Images to Support Determination of
Polycystic Ovary Syndrome." In Journal of Physics: Conference Series,
vol. 622, no. 1, p. 012027. IOP Publishing, 2015.
[7] Setiawati, E., and A. B. W. Tjokorda. "Particle swarm optimization on
(a) follicles segmentation to support PCOS detection." In Information and
Communication Technology (ICoICT), 2015 3rd International
Conference on, pp. 369-374. IEEE, 2015.
[8] Wisesty, Untari N., and Jondri Nasri. "Modified Backpropagation
Algorithm for Polycystic Ovary Syndrome Detection Based on
Ultrasound Images." In International Conference on Soft Computing and
Data Mining, pp. 141-151. Springer, Cham, 2016.
(b) [9] Wisesty, Untari N., and Jondri Nasri. "Modified Backpropagation
Fig. 9. (a) The Interest Area of Filters in Layer 3 and (b) Interest Area After Algorithm for Polycystic Ovary Syndrome Detection Based on
Pooling. Ultrasound Images." In International Conference on Soft Computing and
Data Mining, pp. 141-151. Springer, Cham, 2016.
After Layer 4 the data are reshaped into single vector for [10] Ciresan, Dan Claudiu, Ueli Meier, Jonathan Masci, Luca Maria
Gambardella, and Jürgen Schmidhuber. "Flexible, high performance
fully connected layer and scoring layer. The overall result of this convolutional neural networks for image classification." In Twenty-
network is very good, this proposed approach can achieve Second International Joint Conference on Artificial Intelligence. 2011.
performance of 100% micro-average F1-score with average of [11] Masko, David, and Paulina Hensman. "The impact of imbalanced training
76.36% on test data. data for convolutional neural networks." (2015).
[12] Srivastava, Nitish, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever,
and Ruslan Salakhutdinov. "Dropout: a simple way to prevent neural
networks from overfitting." Journal of Machine Learning Research 15,
no. 1 (2014): 1929-19
Authorized licensed use limited to: J.R.D. Tata Memorial Library Indian Institute of Science Bengaluru. Downloaded on April 06,2025 at [Link] UTC from IEEE Xplore. Restrictions apply.
