Built a Computer Vision engine to detect anomalies (darker or lighter spots) in thermal photos of solar batteries

Client

A solar energy company from the United Kingdom.

‍

Problem

Solar panels are gaining more popularity in various sectors thanks to the benefits they offer. They reduce ever-rising electricity costs and help protect the environment. Despite being easy to maintain, solar panels wear out due to weather and mishaps. The silicon semiconductors react to the temperature and humidity changes and can degrade even with regular cleaning.

Our client faced the problem of having to inspect many solar panels and watch their electrical input regularly to avoid failures. It can be quite a daunting task, especially since they had plenty of them to observe.

Solution

As the size of the image dataset was small and unlabeled, we decided to proceed with classical Computer Vision methods. With a larger and labeled dataset, a simple object detection model would be suitable for this case. The main challenge would be to distinguish real failure cases from other possible sources of occlusion (e.g. a bird flying over) and from a small light square that each cell has that can be confused with a failure. Given enough variable data, this would be learned automatically, maybe requiring some handcrafted heuristic result postprocessing.

Nevertheless, the solution was designed to consist of four stages. All parameters were selected for the particular combination of image size and image quality and might be adapted if any of these changes.

Stage I

Applied to the entire solar panel image:

• Transform image to grayscale.
• Use a bilateral filter with filter size 9 to remove noise while keeping the edges sharp.
• Apply Otsu binarization which automatically chooses the best suitable threshold to obtain a binary image.
• Use morphological opening with a kernel size of 51 to smoothen the borders of the plates and remove left noise artifacts.
• Find contours on the obtained image to extract panel regions from it

Stage II

For each solar panel crop:

• Extract panel region from the original image
• Detect contours on this panel using all hierarchy of contours detected with cv2.findContours
• Keep contours with the area bigger than 100 pixels to include even partially seen panels
• Rotate the panel to make cells aligned with the axis
• Extract cell regions from the panel region

Stage III

For each detected solar cell:

• Enlighten blackened areas by enhancing their color in order to get rid of unwanted black cells occasionally extracted at the previous step — especially important when the panel is heavily declined from horizontal/vertical lines
• Apply adaptive histogram normalization (CLAHE)
• Apply gamma adjustment in order to flatten the brightness of the overall matrix
• Use Gaussian blur to remove noise
• Perform Otsu binarization to obtain a binary image
• Apply erosion to remove white noise
• Detect contours on this panel using all hierarchy of contours detected with cv2.findContours
• Keep contours with an area larger than 250 pixels

Stage IV

Detecting anomalies:

• Filter out the cells with the area out of the defined interval.
• Using a priori information about regular hot spot positions, remove them from consideration by not considering the first 10 pixels from the smaller side.
• Use the Kaze detector from OpenCV with 0.002 as the response parameter to accept the key point on the Otsu binarized cell image extracted from the original image given the contours detected during Stage III.
• Filter out the key points with a small area and the brightness less than the empirically set threshold.

Datasets

The customer provided a small dataset of both RGB and radiometric thermal images. The dataset was not labeled, so there was no opportunity to train a supervised object detection model on it.

The client especially emphasized that they want to detect real failures, not the cases when something/someone (e.g. a bird) occupies some region of solar panels.

Server

Everything was developed and run on a MacBook Air laptop on CPU inference.

CPU: 1,6 GHz Dual-Core Intel Core i5
Memory: 8 GB 1600 MHz DDR3

‍

Results

Our automated AI engine for solar panel monitoring provided the following benefits for the client:

• Reduced downtime by detecting failures quickly.
• Saved on repair costs by implementing a proactive maintenance strategy.
• Reduced the volume of manual work with automated alerts.