License Plate Recognition

The License Plate Recognition project was developed in the context of Vision-Based Systems, a 4th year course in FEUP’s MIEEC. I worked with André Ribeiro, João Oliveira and Leonor Santos, who I hope are doing great in their careers.

The project was divided into two main tasks:

1. Develop an algorithm for the detection of the car plate area. The algorithm should detect and segment the plate and create a region of interest (ROI) containing the detected area. The algorithm should be automatic, i.e., work without any user interaction. Finally, the performance of the detection method is to be evaluated via the Jaccard Index of the segmented ROI.

2. Develop an algorithm to identify the characters (letters and digits) that form the plate. The algorithm should be applied to the plate images that can be obtained from the ground-truth (GT) of the first task. The performance of the method is to be evaluated via the percentage of well recognized chars in each plate.

I was alocated to the second task, so this post will focus on that.

Character detection

After processing the image, the detection of character regions is performed using the regionprops function. The result of the detection is represented in the figure below, where a rectangle is drawn around each region.

In the next step, each of the detected characters is stored in a vector. Each of the characters enters a cycle where it will be compared with all possible characters at its position.

Analyzing all the available license plates, it was noticied that some positions could only be letters or numbers. So, depending on the index of the character in the license plate, it was known from the start whether it would correspond to a number or a letter.

• Indexes 1 and 2 - letters
• Indexes 3, 4 and 5 - number
• Index 6 - letter or number
• Indexes 7 and 8 - letter

Using this knowledge, we can limit the number of comparisons to be made next. Furthermore, characterizing the number of holes present in each character will also allow us to reduce the number of iterations to be performed. For this, we resort to the Euler number, obtained through the function regionprops.

• Euler number = 0 (1 hole)
  • Letters: A, D, O, P, Q, R
  • Numbers: 0, 4, 6, 9
• Euler number = 1 (0 holes)
  • Letters: C, E, F, G, H, I, J, K, L, M, N, S, T, U, V, W, X, Y, Z
  • Numbers: 1, 2, 3, 5, 7
• Euler number = -1 (2 holes)
  • Letters: B
  • Numbers: 8

Using the example plate above, the first character will only be compared with letters and, within these, only with those with 1 hole, going from 36 possible comparisons (10 numerals + 26 letters) to 6.

The templates were obtained by sampling characters from the license plates under study. For certain letters that are not present in the dataset provided, characters of a typeface with an alphabet very similar to that found in the license plates (Targa) were used as templates.

The implemented template matching method is based on cross correlation. For each template and over each region the function normxcorr2 is applied. The identified character corresponds to the one presenting the template with the highest correlation index. Note that a rescaling of the template to character dimensions is performed before performing the comparison.

Results

With the exception of plate #21, all license plates are correctly identified. The misidentified character corresponds to a zero interpreted as the letter ‘O’. Being at position 6, the algorithm described above will have to go through both digits and letters, being unable to distinguish between the two. The case in question is represented in the figure following figure.

The percentage of correctly identified license plates is $\frac{39}{40} = 97.5\%$.