Sony IMX500 And Sony IMX501 Introduction
Sony Corporation on Thursday announced that it has developed the world’s first image sensor with built-in artificial intelligence (AI) technology. Sony says the AI image sensor will be used in retail and industrial applications and is expected to perform multiple data collection tasks more safely and quickly than existing methods.
Sony’s new IMX500 and IMX501 sensors integrate a logic processor and memory that can convert each module into a microcomputer. The key advantage of this technology is that the chip can perform AI tasks to identify, analyze or count targets of interest without having to share the information with another chip, improving privacy and data security and promising near real-time analysis and target tracking.
Sony’s new sensor is capable of shooting 12-megapixel regular still images and supports 4K video at up to 60 frames per second. Sony says it has sent samples of the new processor to potential customers.
Sony IMX600y – Suggested Reading.
Sony IMX500 and Sony IMX501, these two models of inteligent vision sensors, the principal picture sensors on the planet to be furnished with AI handling functionality. Including AI preparing usefulness on the picture sensor itself empowers fast edge AI handling and extraction of just the fundamental information, which, when utilizing cloud administrations, decreases information transmission inactivity, addresses protection concerns, and lessens power utilization and correspondence costs.
These products expand the opportunities to develop AI-equipped cameras, enabling a diverse range of applications in the retail and industrial equipment industries and contributing to building optimal systems that link with the cloud.
Sony IMX500 And Sony IMX501 Price and Availability
| Model name | Sample shipment date | Sample price (excluding tax) |
|---|---|---|
| IMX500 1/2.3-type (7.857 mm diagonal) approx. 12.3 effective megapixel intelligent vision sensor (bare chip product) | April 2020 | 10,000 JPY |
| IMX501 1/2.3-type (7.857 mm diagonal) approx. 12.3 effective megapixel intelligent vision sensor (package product) | June 2020 (planned) | 20,000 JPY |
The spread of IoT has resulted in all types of devices being connected to the cloud, making commonplace the use of information processing systems where information obtained from such devices is processed via AI on the cloud. On the other hand, the increasing volume of information handled in the cloud poses various problems: increased data transmission latency hindering real-time information processing; security concerns from users associated with storing personally identifiable data in the cloud; and other issues such as the increased power consumption and communication costs cloud services entail.
The new sensor products feature a stacked configuration consisting of a pixel chip and logic chip. They are the world’s first image sensor to be equipped with AI image analysis and processing functionality on the logic chip. The signal acquired by the pixel chip is processed via AI on the sensor, eliminating the need for high-performance processors or external memory, enabling the development of edge AI systems.
The sensor outputs metadata (semantic information belonging to image data) instead of image information, making for reduced data volume and addressing privacy concerns. Moreover, the AI capability makes it possible to deliver diverse functionality for versatile applications, such as real-time object tracking with high-speed AI processing. Different AI models can also be chosen by rewriting internal memory in accordance with user requirements or the conditions of the location where the system is being used.
World’s first image sensor equipped with AI processing functionality
The pixel chip is back-illuminated and has approximately 12.3 effective megapixels for capturing information across a wide angle of view. In addition to the conventional image sensor operation circuit, the logic chip is equipped with Sony’s original DSP (Digital Signal Processor) dedicated to AI signal processing, and memory for the AI model. This configuration eliminates the need for high-performance processors or external memory, making it ideal for edge AI systems.
Metadata output
Signals acquired by the pixel chip are run through an ISP (Image Signal Processor) and AI processing is done in the process stage on the logic chip, and the extracted information is output as metadata, reducing the amount of data handled. Ensuring that image information is not output helps to reduce security risks and address privacy concerns. In addition to the image recorded by the conventional image sensor, users can select the data output format according to their needs and uses, including ISP format output images (YUV/RGB) and ROI (Region of Interest) specific area extract images.
High-speed AI processing
When a video is recorded using a conventional image sensor, it is necessary to send data for each individual output image frame for AI processing, resulting in increased data transmission and making it difficult to deliver real-time performance. The new sensor products from Sony perform ISP processing and high-speed AI processing (3.1 milliseconds processing for MobileNet V1) on the logic chip, completing the entire process in a single video frame. This design makes it possible to deliver high-precision, real-time tracking of objects while recording video.
Selectable AI model
Users can write the AI models of their choice to the embedded memory and can rewrite and update it according to its requirements or the conditions of the location where the system is being used.
For example, when multiple cameras employing this product are installed in a retail location, a single type of camera can be used with versatility across different locations, circumstances, times, or purposes. When installed at the entrance to the facility it can be used to count the number of visitors entering the facility; when installed on the shelf of a store it can be used to detect stock shortages; when on the ceiling it can be used for heat mapping store visitors (detecting locations where many people gather), and the like. Furthermore, the AI model in a given camera can be rewritten from one used to detect heat maps to one for identifying consumer behavior, and so on.
Sony IMX500 And Sony IMX501 Specifications Comparison
| Features | MODEL | IMX500 (bare chip product) | IMX501 (package product) |
|---|---|---|---|
| Number of effective pixels | 4056(H)×3040(V), approx. 12.3 megapixels | 4056(H)×3040(V), approx. 12.3 megapixels | |
| Image size | Diagonal 7.857 mm (1/2.3 type) | Diagonal 7.857 mm (1/2.3 type) | |
| Unit cell size | 1.55μm(H)×1.55μm(V) | 1.55μm(H)×1.55μm(V) | |
| Frame rate | Full pixel | 60fps | 60fps |
| Video | 4K（4056×2288）60fps 1080p 240fps | 4K（4056×2288）60fps 1080p 240fps | |
| Full/video+AI processing | 30fps | 30fps | |
| Metadata output | 30fps | 30fps | |
| Sensitivity (F5.6 standard value) | Approx. 250LSB | Approx. 250LSB | |
| Sensor saturation signal level (minimum value) | Approx. 9610e- | Approx. 9610e- | |
| Power supply | Analog | 2.7V | 2.7V |
| Digital | 0.84V | 0.84V | |
| Interface | 1.8V | 1.8V | |
| Main functions | AI processing function, ISP, HDR shooting | AI processing function, ISP, HDR shooting | |
| Output | MIPI D-PHY 1.2 (4lane)/SPI | MIPI D-PHY 1.2 (4lane)/SPI | |
| Color filter array | Bayer array | Bayer array | |
| Output format | Image (Bayer RAW), ISP output (YUV/RGB), ROI, metadata | Image (Bayer RAW), ISP output (YUV/RGB), ROI, metadata | |
| Package | – | Ceramic LGA 12.5mm(H)×15.0mm(V) |
