Xiaomi’s Night Owl Algorithm
Xiaomi 11 Pro and Ultra debuted with Xiaomi’s self-developed “Night Owl” algorithm, which is claimed to achieve a better photo denoising effect, recover more details and finally get clear and bright images. Today Xiaomi officially introduced this Night Owl algorithm in detail, saying that the algorithm can bring the three advantages of small noise, handheld stability, color accuracy, through innovative deep learning AI algorithm, breakthrough dark light visual limits, to achieve a clear, bright image can still be shot in the environment even if you can’t see your fingers.
Amazing Xiaomi Night Owl Algorithm
There are some photo scenarios that you may not touch 99% of the time: when the ambient light illumination is below 0.1 Lux, it is almost difficult for the human eye to see things, and even keys dropped on the ground are difficult to find, and the photos taken at that time will be completely unusable. But Xiaomi has developed a magical algorithm for this 1% extreme shooting scenario: Night Owl.
What problems do traditional cameras and cell phones encounter when shooting in extremely dark environments?
- Difficulty one, low ambient brightness, high noise, existing deep learning technology is difficult to collect data, it is difficult to effectively reduce noise.
- Difficulty two, the use of traditional algorithms for multiple stacks, can only mitigate the impact of noise and chromatic aberration, it is difficult to recover the lost image information.
- Difficulty three, the use of long exposure method, through the growth of exposure time to improve image quality, the user handheld shooting is easy to paste into a piece.
- Fourth, the traditional brightening algorithm will significantly increase the difficulty of de-noise, but also affect the white balance resulting in the loss of accuracy of color correction.
For the above problems, Xiaomi 11 Pro and Ultra first self-researched Night Owl algorithm, by obtaining eight consecutive correctly exposed EV0 photos, combined with self-researched image metering algorithm, image alignment fusion algorithm, image reconstruction algorithm, image color correction algorithm, to achieve better denoising effect, restore more details, and finally get clear and bright images.
The biggest difficulty of deep learning technology is the acquisition of training data. Considering the low brightness and high noise of an extremely dark scene environment, it largely increases the difficulty of data acquisition.
Low Noise: the image is pure not like a night scene photo.
Xiaomi’s Night Owl algorithm independently developed a noise calibration system for extremely dim light scenes, fully understanding the distribution and shape of noise in extremely dim light scenes, thus supplementing a large amount of simulated noise data, enhancing the richness of training data, and making the denoising process more targeted; at the same time, the Night Owl algorithm independently developed a data collection system for extremely dim light scenes, using a variety of real cameras for data collection and supplementing real camera data to obtain better denoising effects.
Handheld stability: achieve the effect of tripod long exposure 30 seconds stable.
The biggest characteristic of the very dark light scene is the noise, first of all, the biggest difficulty is how to effectively remove the image noise at the same time, to recover more details of the image. At present, even with the use of multi-frame image overlay denoising, there will be image edge information loss and partial content distortion. The neural network can be optimized for image denoising and utilizing deep learning, it can recover high-quality local images rich in details even if there is local information mutilation.
The AI denoising algorithm in the Night Owl algorithm combines multi-frame EV0 RAW domain image information to perform image alignment and reduce the impact of shooting handshake; it can also achieve full use of intra-image information and inter-image information to complement each other, to obtain better denoising effects and recover richer detail information.
Color accuracy: AI metering and color correction module solves the problem of color distortion.
Brightness enhancement and color restoration are also major challenges for extremely dark light scenes. Performing excessive image brightening will lead to increased difficulty in denoising, while a large amount of noise will cause the traditional white balance process to fail to obtain accurate key information points, resulting in color correction misalignment.
The Night Owl algorithm also includes AI metering and color correction modules, which can calculate the required exposure parameters, frame rate, and color information based on the scene information and sensor information, and realize different degrees of brightness enhancement and color restoration process according to different environmental brightness and scene content, weighing the impact degree of brightening and noise removal, effectively solving the color distortion problem and improving the environmental self-adaptability of the Night Owl algorithm. The Night Owl algorithm can adapt to the environment.
The Night Owl algorithm breaks through the limits of dark light vision through an innovative deep learning AI algorithm, enabling clear, bright images to be captured even in environments where no one can see their fingers.