Microsoft AI Repair Program and In-display Camera
After the success of perforated front cameras, the next big thing for phone front cameras development is an in-display camera, where the camera is behind a transparent OLED screen, which is not in use. Still works as a normal display, but allows enough light to pass through to allow the camera below to capture a usable photo.
Visionox Under Screen-Camera World’s First Mass-producible Solution – Related Reading.
Microsoft’s research department has been looking at how off-screen cameras take usable photos, and they note that diffraction of the screen’s pixel structure will be blurring the image, reducing the contrast, reducing the available illumination, or even blocking some of the image content completely, depending on the device’s display pixel design.
Fortunately, this happens predictably, and because the pixel structure generally occurs only horizontally. To compensate for the image degradation inherent when shooting through a T-OLED screen, the researchers used a U-Net, the neural network structure both improves the signal-to-noise ratio and eliminates image blurring. The team was able to achieve recovered images that were virtually indistinguishable from those taken directly.
The improvements can be seen in the following sample image.
Microsoft’s research into this work appears to be primarily for use on the big screen in video conferencing but believes it is equally for smartphones, more information on this Microsoft project can be found here.