Case Study 2: Automatic Number-Plate Recognition

As the name implies, automatic number-plate recognition (ANPR) is a technology that can recognize and extract the license plate number from one or more images of a car. There are numerous practical applications of ANPR such as identifying cars, entering a controlled parking lot or gated community, automated toll collection, traffic violation capture, and more.  

The core technology behind ANPR is Optical Character Recognition (OCR), the system that converts images to editable and searchable data. However, there is also a good amount of pre-processing required to deal with some of the many challenges of ANPR before we can apply OCR.

Challenges

Some of the challenges with ANPR technology include the following:

  1. Sometimes low-resolution images are captured because of the camera or the distance between the device and the plate. These scenarios and low-contrast images due to lighting conditions can reduce OCR accuracy.  
  2. Since license plates are different between countries and regions, there can be unique sequences of letters or fonts that are trickier to recognize.
  3. Errors in license plate recognition can occur when there is variation in color, screws, or other adornments around the plate (e.g. borders).
  4. In the case of real-time ANPR, processing speed can introduce challenges too.

Solution

Although ANPR technology has been around for many decades, some of the traditional algorithms have been replaced by newer deep learning algorithms in order to improve these challenges. Now with modern ANPR systems, publicly reported accuracy is in the 93-95% range.

With the StreamLogic platform, we have seamlessly integrated deep learning models while still leveraging traditional image processing in a number of pre-processing steps. As a result, we have created a solution that builds complete end-to-end computer vision applications with higher accuracy rates.

Understanding the Technology

To combat the challenges of traditional ANPR systems, we applied some key techniques.

Step 1: Detect License Plates

First, we want to quickly identify the location of the license plate within a larger image and extract only that specific region to apply OCR to. This task is best accomplished with Convolution Neural Networks (CNN), a deep learning algorithm that takes an input image, and can detect and locate objects based on their unique aspects.   

However, publicly available CNN models for license plate detection tend to be relatively slow when compared to other object detection models like YOLO. To resolve this issue, we found that cascading YOLO and license plate detection CNN improves overall processing time. So we apply the YOLO model first to the full camera image to detect cars. After this is accomplished, the license plate detection model is then applied only to that region detected by YOLO.

Step 2: Improve the Quality of OCR Results

We have experimented a great deal with applying filters to low-resolution images to improve the quality of OCR results. OCR technology is designed for binary images, and conversion to a binary image is usually the final step before OCR is applied.

The challenge is that with low-resolution images, it can be difficult to extract the license plate because the letters often run together or bleed into surrounding adornments like borders or emblems due to insufficient pixel space between them.

In order to mitigate this issue, we use the following approach:

  1. Apply a sharpen filter
  2. Upscale the image (4x)
  3. Apply a small blur (5x5)

This technique has proven useful in recognizing plates as small as 16 pixels high.

Step 3: Convert Image to Binary and Conduct Cleanup

We then want to convert the image to binary and apply a cleanup step to remove unwanted artifacts such as screws or borders. Conversion to binary is performed using the fairly standard OTSU algorithm, a method used to perform automatic image thresholding.

Cleanup then involves analyzing the remaining visual components and removing anything that is not likely to be a character. For example, looking at the width to height ratio, one can easily eliminate horizontal/vertical lines like plate borders.

Step 4: Apply OCR

Finally, it’s time to apply OCR. OCR technology has been around for a long time and was based on more traditional image processing techniques of localizing characters, extracting features, and identifying specific characters. This traditional approach has given way to the newer deep learning approach with better accuracy which improves ANPR. It is also fairly easy to generate ample training data for these algorithms to produce robust models. These means they can be easily trained for any specific font or multiple fonts.

Summary

By putting these techniques into practice, the StreamLogic platform leverages both newer deep learning algorithms and traditional image processing algorithms to improve the accuracy of ANPR and provide a platform to build complete end-to-end computer vision applications.