Objective and unbiased image quality analysis of smartphone and other mobile device cameras.
To choose the smartphone/tablet camera that suits you best, check out which devices are on top of the game.
VCX Score Update Version 1.5
We are pleased to announce the update of the VCX Score.
In order to reflect the user experience better and to make the score of the different shooting condition comparable and more intuitive, the VCX score was revised and has been updated as of 22.12.2016. The update leads to a lower score for all devices compared to version 1.0. As technologies are constantly improving, this also enables keeping the VCX score valid for a longer period of time.
The most important changes are:
- Changed weighting distribution of the main shooting conditions (bright, mid, low, flash, zoom).
- Clear separation between the main shooting conditions by taking “pinch zoom” out of the VCX score
- Consistent weighting throughout the main shooting conditions in order to make the individual scores for each shooting condition more intuitive and comparable.
- Revision of formulae, including introduction of formulae that lead to point deduction in case the result is way below user acceptance (resolution and texture related parameters only)
- Correction and adjustment of the whitepaper
If you would like more information about the score in general as well as information about the changes of the update, please request the whitepaper.
The objective and comprehensive score for mobile devices
- The camera is one of the most important components of mobile devices
- The score is based on image quality (70%) and performance (30%)
- The testing is accomplished by an independent test lab
- The evaluation is automated and 100% objective
- The weighting of image quality aspects has been predefined by experts and is applied to all devices in the same way
- All tested devices are bought off the shelf (no selected samples are accepted)*
- All devices are tested under multiple illumination levels and use cases
Due to the fast evolving technologies and the ever increasing importance of Social Media, the camera is one of the most important components of today’s mobile phones. Nowadays, smartphones are taking over a big share of the compact camera market. A simple reason for this might be revealed by the famous quote of Chase Jarvis: “The best camera is the one that’s with you”.
But with the vast choice of devices and great promises of manufacturers, there is a demand to characterize image quality and performance in very simple terms in order to provide information that helps choosing the best suited device.
The result of this is the VCX score (Valued Camera eXperience) that shall reflect the user experience regarding the image quality and the performance of a camera in a mobile device.
The VCX Score
The VCX score is generated based on 100% objective data. The objective data is the result of well-defined and transparent test procedures following international standards where possible. The entire analysis is only based on the captured images and the analysis algorithms applied to these images. Based on a fixed algorithm, the score is calculated using the numerical results. So in comparison to other metrics on the market, this metric is not modified based on the subjective impression of only a few people.
Neither special samples from suppliers nor custom hardware/software are accepted. The results are obtained from devices that are launched onto the market.* The devices are tested in auto mode using the default camera application and setting in order to reflect the normal use of an average consumer.
The image quality is evaluated for five different use cases “Bright” at 1000lux, “Mid” at 250lux, “Low” at 63lux, “Flash” at 63lux with the flash activated and “Zoom” at 1000lux with 4x digital zoom (if available), covering the most important aspects like:
- Spatial Resolution – What level of details can I see?
- Texture loss – How does the device reproduce low contrast, fine details?
- Noise – How much disturbing noise do I see?
- Dynamic range – What is the maximum contrast in a scene the device can reproduce?
- Color Reproduction – Are there any issues in the color processing?
For the main image quality analysis, the T42 multi-purpose test chart is used, ensuring that all measurements are performed under exactly the same condition with exactly the same camera parameters.
Additionally, the user experience is driven by the responsiveness of the device, so we check how fast it can focus and capture images.
As mobile phone cameras are devices that are used in any condition and mainly hand-held, the low light performance is very important. We check how much the image quality changes in low light, depending on the movement of the device.
Calculating the VCX score:
The VCX score is calculated based on objective, numerical results. It does not contain any visual assessment or other subjective components.
The only subjective component is the weighting, meaning the decision which metric is more important for the overall performance compared to others. But this weighting has been accurately determined by a group of experts and it is identical for every device, so the comparison between devices is fixed and not influenced by individual opinions.
The total score range is between 0 and 100. The range is designed in a way, that a value of 100 means that the device meets the best possible result in every metric that is achievable with today’s camera technology. So it can happen that in the future the value of 100 is exceeded or that the range has to be modified. With the continuously evolving technologies, it is likely that there will be a need for updating the test procedures as well as the weighting of the scoring system approximately every two years.
VCX Weighting distribution
VCX has been developed over several years. Vendors from the mobile device and chipset industry have contributed positively to its improvement and this cycle of feedback from customers and vendors shall continue in order to achieve the most accurate results that reflect end-to-end user experience.
To obtain detailed information about the VCX score (test procedure, metrics, formulae, etc.), you can request the Whitepaper here.
If you would like to share your opinion or have any questions, please use the comment box below or use the contact form at the bottom of the website.
©Copyright: The whitepaper and its content are governed by the copyright laws of Germany. Any duplication, processing, distribution or any form of utilization requires the prior written consent of the author or authors in question.
* If a need arises due to market forces and/or user demand that a device result be published at launch under the VCX umbrella, the result will be clearly marked as “provisional” beside the VCX logo.