OPPO FDF Full-dimensional Portrait Video Technology
On November 18, OPPO unveiled the OPPO FDF full-dimensional portrait video technology system for the first time at the Future Technology Conference 2020 (OPPO INNO DAY 2020) with the theme of “Leap Into The Future”. The system is composed of two engines: portrait perception and image quality enhancement, which are optimized for the human subject and the overall picture during portrait shooting respectively. It is reported that the first products equipped with the system will be available in December.
OPPO introduces the FDF full-dimensional portrait video technology system, which is composed of two engines: the portrait perception engine and the image quality enhancement engine, which are optimized for the human subject and the overall picture during portrait shooting.
Specifically, the perceptual portrait engine has AI face detection, AI super-clear portrait, OFL skin optimization algorithm, face distortion correction, and other functions to reject uniformity and make the works more natural, realistic, and touching.
By optimizing the network model, AI face detection achieves 99.78% detection accuracy and 25ms single-frame detection time, so it can quickly detect the face area for video or photo shooting, and set a precise range for subsequent face optimization.
Meanwhile, the AI algorithm marks 296 key points in the face area to distinguish facial features and contour details and adds auxiliary information such as whether the key points are visible or not, and the face angle, etc. Finally, it achieves an NME as low as 4.71% on the authoritative WFLW dataset, and the running frame rate of the algorithm on the mobile platform reaches over 200fps.
The AI Ultra Clear Portrait is designed with three sub-modules in the deep neural network model to perform over division calculation, noise reduction optimization, and sharpness optimization on the face. The algorithm accurately detects and enhances the face and adjacent areas, even in low light conditions.
The OFL Skin Optimization algorithm provides AI processing for skin tone and skin type, respectively, to help remove facial imperfections, optimize skin performance, and enhance the overall appearance of the user’s face when shooting. The algorithm achieves 96.5% segmentation of skin areas and 97.4% facial skin accuracy. The “skin tone detection” covers not only the face but also the neck and hands, covering almost all the areas of skin that are exposed during portrait videos.
The OFL skin optimization algorithm also enables AI blemish removal, which intelligently identifies and distinguishes between natural and acquired features for a more natural blemish removal effect. Acquired imperfections such as spots, acne, and bags under the eyes are removed, and natural facial features such as moles are preserved.
Related Reading.
In facial aberration correction, the background part of the perspective projection is taken, and the face area is cut and separated, and a spherical projection is made again so that the face maintains its normal shape and conforms to the human eye’s viewing habits.
The Image Enhancement Engine includes a new generation of Super Image Stabilizer, Motion Timer, Super Night Scene Video, DOL-HDR, and other functions to ensure satisfactory shots in scenes with high screen shake, low light, and excessive brightness.
The Super Image Stabilization algorithm adds a scene detection process to the gyroscope, accelerometer, and Sensor data to determine motion trajectory, attitude, and exposure to distinguish different scenes and match different image stabilization strategies.
For handheld mobile time-lapse photography, OPPO adds motion trend judgment to the Super Image Stabilizer algorithm, which not only makes the transition between extracted frames smoother but also achieves longer time stabilization, making the real-time performance of mobile time-lapse photography almost reach the gimbal effect.
Based on the previous generation algorithm, Super Night Scene video is integrated with the HDR algorithm to differentiate and debug the brightness adjustment curve. The algorithm will first intelligently determine whether to enable the Night Scene or HDR brightness adjustment curve based on the current brightness environment to achieve a more transparent and better video effect.
DOL-HDR solves users’ backlight shooting pain points, and its long and short exposure frames are almost simultaneous, significantly reducing the interval between before and after frames, greatly improving ghosting problems, and achieving better HDR video shooting results.
OPPO said that the first new products equipped with OPPO’s portrait video technology system will be available in December. At present, OPPO’s technologies in HDR, night scene, super stabilization, etc. have been made available to third-party platforms such as Shakespeare and Qiaoshou. Stay tuned via Twitter for further updates.