Mobile HDR Photography on Android

This was my final project for CS 231M Mobile Computer Vision, in the Spring of 2014.  Everyone in the class was given a Tegra Note 7 Android tablet for the quarter, so that's the device I used during this project.

High Dynamic Range (HDR) imaging is a process by which we can artificially increase the dynamic range of an image, in order to produce a more visually appealing image, with fewer under- or over-exposed pixels.  This way, both bright and dark regions of a scene can be made visible in an image, and the apparent dynamic range will seem more similar to that of human vision, rather than the limited dynamic range of camera sensors.  Software-based approaches to HDR imaging involve taking multiple shots at different exposure values, to capture the large dynamic range of the scene, and then performing a tone-mapping of some sort so that the final image can still be represented in the standard 8-bit format.  The two stages are often referred to as bracketing (the selection of exposure times and capturing of those multiple shots) and exposure fusion (the combining of those multiple shots into a single HDR image).

Below is an example of an HDR image produced by 3 shots, taken in quick succession, with different exposures.  See how the HDR image is more visually appealing, and how all regions of the scene seem well-illuminated.

A lot of the recent Android phones/tablets, as well as iPhones, come with an HDR camera mode, which is in part a testament to how useful and appealing this method is to photography, and to how desirable it is to have on mobile devices, whose camera sensors often saturate easily.

My project was to implement the bracketing and exposure fusion stages from two recent papers on mobile HDR imaging, and compare all the combinations of bracketing and exposure fusion stages in terms of runtime performance and image quality.  I wasn't trying to propose a new HDR method, since that seemed a bit too much to bite off at the time, but this analysis-type project allowed me to gain experience in implementing realistic HDR methods and learn about the trade-offs between different approaches.

My report and project code are linked.  I used the FCam library, which is unfortunately only available on the Nokia N900 and recently on the Tegra Note, to gain precise control over the device's camera module.  I also built my HDR imaging modes on top of the Nvidia FCameraPro sample app.

Below is an example of 4 HDR images of the same scene, produced using the 4 combinations of the bracketing and exposure fusion methods proposed by the two papers.  The histograms at the bottom are the pixel intensity distributions for each of the 4 HDR images.

In conclusion, I got to implement two sets of pretty different HDR methods on an Android device, get the running in a practical way, and analyze the advantages and disadvantages of the different methods.  I had to work through the limited memory available on the mobile device, which was fun and frustrating at the same time.  And of course, I learned a lot about HDR imaging and mobile device programming.