Description

Audio processing, Video processing and Computer Vision Lab exercise 3

1. Introduction
This field of microscopy image analysis is attracting a lot of attention as a tool to facilitate a quantitative automatic analysis. In this lab exercise, we focus on the segmentation step.
In particular, the goal is to produce a segmented image as illustrated in Figure 1, where the image on the left shows the image to be segmented and the image on the right shows the desired result.

Figure 1.- Original and ground-truth segmentation result.
2. Project description
The basic block diagram of the automatic image segmentation system is as follows:

Figure 2.- Block diagram of the segmentation process
1. Preprocessing: to reduce the impact of noise or other artifacts
2. Segmentation: use of an automatic segmentation algorithm to separate each cell, generating a label image (as many labels as cells).
3. Post-processing: to achieve a more precise segmentation, the binary masks resulting from the segmentation can be post-processed (removing holes within the cells, separating several cells segmented as one, etc.)
2.1. Preprocessing
A Gaussian smoothing operation is a common option, but you need to pay attention to the tradeoff between the blurring of undesirable features and the preservation of the edges. Median filtering is another option.
2.2. Automatic segmentation
A wide variety of methods have been proposed for automatic cell segmentation. In class, we have studied the most basic approaches, namely:
1. Threshold-based segmentation: these algorithms determine one or more appropriate threshold values to separate the pixels of an image into two or more regions.
2. Clustering-based segmentation: these methods segment an image into two or more regions according to a set of predefined features (intensity, texture, location, etc.).
2.3. Post-processing
The result of the automatic segmentation stage is a binary mask that separates the area of each cell from the background.
In order to improve the result of the segmentation, it can be postprocessed by applying one or more morphological operations: erosion, dilation, opening, closing, hole filling, etc.
3. Evaluation
The metric used for the evaluation of the proposed segmentation system will be the Jaccard Index (𝐽𝐽𝐽𝐽). It measures the degree of similarity between two regions, in this case two segmentation masks:
– the ground-truth (GT) mask, determined by experts – the mask predicted by our automatic system.
The formulation of the 𝐽𝐽𝐽𝐽 is as follows:
|𝐴𝐴 ∩ 𝐵𝐵|
𝐽𝐽𝐽𝐽(𝐴𝐴, 𝐵𝐵) =
|𝐴𝐴 𝑈𝑈 𝐵𝐵|
i.e., the cardinality of the intersection of both sets divided by the cardinality of their union. Given a segmentation system, 𝐴𝐴 would represent the GT segmentation mask and 𝐵𝐵 the predicted mask (or vice versa).
𝐽𝐽𝐽𝐽 takes values between 0 and 1. The closer to 1, the better the segmentation.
4. Database
– The database provided for this exercise consists of 40 images of size 1024x 1280, in TIF format, taken from [1]:
https://www.ebi.ac.uk/biostudies/studies/S-BSST265
– There are two types of images:
o The images themselves: intensity images
o The ground-truth segmentation: label images (one label per cell)

5. Goal
You have a baseline segmentation system available. Your job is to improve it.
You have many options to try out. Some examples follow:
− Exploring preprocessing techniques
− Clustering-based segmentation
− Morphology-based post-processing
The key point is to improve the system by incorporation those ideas that work well and keep track of the improvements in a Table.
6. Implementation and submission
Your code must implement the different stages of the system: pre-processing, automatic segmentation and post-processing. The software has to be properly documented.
The headers of the provided functions cannot be modified, as they will be used for evaluation. Your system has to work just updating the folders containing the images and the masks.
6.1. Project submission
For the evaluation of the project, you must submit:
1. The software implemented according to the Python scripts provided, properly structured and commented. If you have used some source of external software, you have to include it in your submission (and properly describe it in the project report).
2. A brief project report (3 pages maximum, excluding the cover), including: – Brief introduction (one paragraph; it is just a formality)
– Table including each proposed technique and the obtained results. The idea is starting with the baseline results (provided with the lab exercise) and adding new entries indicating the techniques that allowed you to improve the baseline results. Example:
Technique Performance (IoU)
Baseline system 0.685
Technique #1 0.69
Techniques #1 and #2 0.7
Techniques #1, #2 and #3 0.72

− A brief description of the proposed techniques. The student must provide relevant information about the system (excluding the description of the known methods or algorithms – those than can be found in papers and books). In other words, you only need to describe what you have done without describing the underlying concepts and algorithms. For example: you do not have to describe k-means, but you need to describe how you decide which cluster represents the cells. Example:
“Remove_small_holes (morphological operation): filling the holes improves the segmentation masks, increasing the IoU”
– A sequence of images illustrating the image evolution, from the original image to the segmented image. This image sequence must illustrate the effect and contribution of the proposed technique. Example:
Morphology operation: remove_small_holes

– Discussion of results
– References
6.2. Evaluation
The evaluation of the lab is structured as follows:

Departamento de Teoría de la Señal y Comunicaciones

– 60%: performance
– 30%: report
– 10%: code
6.2.1. Original contributions
All the original contributions will be valued positively.

7. References
[1] Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A. A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology. IEEE Trans Med Imaging. 2017 Jul;36(7):1550-1560. doi: 10.1109/TMI.2017.2677499. Epub 2017 Mar 6. PMID: 28287963.