An Image histogram can be a useful tool when evaluating the exposure of your photographs. A quick look at the histogram after you took a photo might help you find out, if your photo is overexposed or underexposed.
Photography is a really interesting topic for me and I really would like to learn more about it. Not only to improve my photos. I'm also interested in the technical aspect and I find that combining it with Android and some programming experiment would be a good opportunity to learn more about this topic. I'm still a newbie, sot there's a lot to learn.
In this blog post I'll show you how to generate a histogram on Android. In the first part I'll show how to generate the actual values that are displayed in a histogram. I'll show how to do this with Kotlin, C++, and RenderScript.
The second part will show how to visualize the values using a custom View.
You can find the complete code related to this blogpost on github
What's the Data Shown in Histogram
Histogram normally shows the brightness distribution of image pixels. In our case we can think of brightness as intensity of RGB values.
When creating a histogram, what we're basically doing is to separate pixels into groups (bins) of similar brightness levels, and then displaying the size of each group.
On the horizontal axis you show the brightness range from completely dark to completely white. Usually the left side starts with black and goes gradually to white.
The vertical axis shows the number of pixels at each level.
Let's imagine that the picture above is a 4x4 pixel subsection of a photo and we want to calculate histogram values only for this smaller section. In this case we would have 4 bins for the red color intensity - 20, 100, 150, and 255. The bin for 20 (x axis) would contain 5 pixels (y axis), the bin for 100 would contain 3 pixels, and so on.
We can generate a histogram for each of the Red, Green, and Blue values. Or we can also create histogram for greyscale values.
There are multiple methods to convert RGB values that you get from Bitmap to greyscale. In my example I used this simple formula - Greyscale = 0.299R + 0.587G + 0.114B. The green component has the highest weight here because human vision is most sensitive to green. So an alternative to calculating greyscale values when generating a greyscale histogram would be to just take the green component.
Calculating Histogram Values With Kotlin
I will be using Android's Bitmap class to work with the pixels.
You will normally try to get the RGB values for each pixel, so that you can do the calculations. Your images can be in various formats, but you ideally want to work with something specific and predictable.
I loaded an image with decodeFile
bitmap = BitmapFactory.decodeFile(imgPath)You can specify additional options, which allow you to give a hint to the decodeFile() method on which format to load. The default format is ARGB_8888 which should also suit my situation.
To get pixels from Bitmap and extract the RGB values, you can do the following
val pixels = IntArray(bitmap.width * bitmap.height)
bitmap.getPixels(pixels, 0, bitmap.width, 0, 0, bitmap.width, bitmap.height)
for (pixel in pixels) {
val red = Color.red(pixel)
val green = Color.green(pixel)
val blue = Color.blue(pixel)
val luma = (0.299f * red + 0.587f * green + 0.114f * blue).roundToInt()
}I also tried to convert the RGB value to greyscale with a simple formula. As mentioned earlier, there are multiple way to do this, and I just picked this formula because it was simple and efficient to calculate.
You can now use the method above to loop through the RGB values and do the actual histogram calculation
val redBins = IntArray(256) {0}
val pixels = IntArray(bitmap.width * bitmap.height)
bitmap.getPixels(pixels, 0, bitmap.width, 0, 0, bitmap.width, bitmap.height)
for (pixel in pixels) {
val red = Color.red(pixel)
redBins[red]++
}You know that the R, G, and B pixel values will always be in the range 0 to 255 (included). This fact can be used to calculate the histogram relatively efficiently. I pre-allocated an IntArray that will store the histogram values for red. Then I used the red intensity value as an index and just incremented the count for each intensity of red.
After performing the loop above, you basically have all your values for histogram.
One additional thing you might want to do is to normalize the values, so that they are pushed into a specific range, before they can be displayed on a graph. This will allow you to display curves for red, green, blue, and luma histogram values next to each other on the graph with more comparable sizes.
val normalized = IntArray(256) {0}
val min = redBins.min().toFloat()
val max = redBins.max().toFloat()
val newMin = 0.0
val newMax = 255.0
for (v in redBins) {
normalized[i] = ((v - min) * (newMax - newMin)/(max - min) + newMin).roundToInt()
}In the code above I used the regular linear normalization formula to force the values to range 0 to 255.
Generate Histogram in C++ (NDK)
I also tried to do generate histogram using C++/NDK.
In this case there might be no performance difference. But it still can be useful to know how to access Android's Bitmap pixels in native code and how to pass result back to Kotlin (for example if you want to do more processing in C, or use native code with OpenCV or other third-party C libraries).
To make this work, first update your CMakeLists.txt file
cmake_minimum_required(VERSION 3.4.1)
file( GLOB app_src_files
"src/main/cpp/*.cpp"
"src/main/cpp/*.cc" )
add_library( pixel-proc SHARED ${app_src_files} )
find_library( graphics-lib jnigraphics )
target_link_libraries( pixel-proc ${graphics-lib} )You need to link to the jnigraphics library because it contains the native functions that can work with a Bitmap.
If you didn't create your project with C++ support, you might need to create the CMakeLists.txt file manually inside of your Android Studio project module's app folder.
You should also make sure that it is added inside of your module-level build.gradle file
...
android {
defaultConfig {
...
externalNativeBuild {
cmake {
cppFlags "-std=c++1z -fexceptions"
arguments "-DANDROID_STL=c++_shared",
"-DANDROID_PLATFORM=android-23"
}
}
ndk {
abiFilters 'armeabi-v7a'
}
}
...
externalNativeBuild {
cmake {
path "CMakeLists.txt"
}
}
}I also like to pass some additional arguments that affect the build (for example STL or ABI).
My Kotlin code calls an external JNI function that calculates the values in C++ code
class Histogram {
...
companion object {
@JvmStatic private external fun generate(bitmap: Bitmap, redBins: IntArray, greenBins: IntArray,
blueBins: IntArray, lumaBins: IntArray, normalize: Boolean = true)
init {
System.loadLibrary("pixel-proc")
}
}
}The method takes Kotlin's pre-allocated IntArray (int[] in Java) and fills it with histogram values. Android NDK provides methods to access Bitmap functionality directly in C++ code, so I just pass the bitmap as the first argument and continue the processing directly in C.
In C++ code I called AndroidBitmap_lockPixels() to access the pixels of the Bitmap that was passed from Kotlin. I stored the final values in pre-allocated int arrays
#include <jni.h>
#include <android/bitmap.h>
extern "C" {
JNIEXPORT void JNICALL
Java_eu_sisik_pixelproc_histo_Histogram_generate(JNIEnv* env, jclass, jobject bitmap,
jintArray redBins, jintArray greenBins, jintArray blueBins, jintArray lumaBins, jboolean normalize) {
AndroidBitmapInfo info;
AndroidBitmap_getInfo(env, bitmap, &info);
// Check format
if (info.format != ANDROID_BITMAP_FORMAT_RGBA_8888) {
// Handle error here
}
// Access the pixels
void* pixels;
AndroidBitmap_lockPixels(env, bitmap, &pixels);
int* rBins = (int*)env->GetPrimitiveArrayCritical(redBins, 0);
for (int y = 0; y < info.height; y++) {
uint32_t * line = (uint32_t *)pixels;
for (int x = 0; x < info.width; x++) {
int red = (int) (line[x] & 0x0000FF);
int green = (int)((line[x] & 0x00FF00) >> 8);
int blue = (int) ((line[x] & 0xFF0000) >> 16);
rBins[red]++;
...
}
pixels = (uint8_t *)pixels + info.stride;
}
env->ReleasePrimitiveArrayCritical(redBins, rBins, 0);
AndroidBitmap_unlockPixels(env, bitmap);
}
} Note that you can use AndroidBitmap_getInfo() to get some more information from the bitmap, including the format. I assumed ANDROID_BITMAP_FORMAT_RGBA_8888 here (should be equivalent to ARGB_8888 in Java).
Calculate Histogram Values in RenderScript
RenderScript is an additional way to further improve the performance of your image processing functions. This technology can be used to perform parallelized processing on your Android device.
It's possible to use it directly from C++ (NDK), but for this post I only played with the Kotlin/Java interface.
I think the official documentation explains pretty well how to use it. I'll only quickly summarize some main points and show how to implement the stuff relevant to my situation.
RenderScript language is derived from C99 and the functions that you call to perform parallel calculations are called compute kernels. These compute kernels operate on Allocations.
Allocations allow you to pass data that should be processed in parallel in between your Kotlin code and your RenderScript code. You can create allocations from various data - in our case I used IntArrays and the actual Bitmap pixels.
To be able to use RenderScript, you first need to update your module-level build.gradle file
android {
...
defaultConfig {
...
renderscriptTargetApi 24
renderscriptSupportModeEnabled false
}
}I used a relatively high renderscriptTargetApi level in this case (24). The reason is that I wanted to use reduction kernels which are only available since Android 7...
There are two types of compute kernels in RenderScript - mapping kernel and reduction kernel. Mapping kernel can return a processed Allocation. You could look at this as transforming arrays of input data into an array of processed values. Reduction kernels still operate on input Allocations, but return only one single value as the final result of parallel computation. I'll show a simple example of reduction kernel at the end of this section.
Once we configured our build system, we can create an .rs file in our project directory. I needed to place the .rs file into src/main/rs directory, so that the build system could pick it up. My current version of AndroidStudio also offers the option to create the rs directory via File->New->Folder->RenderScript Folder, but I guess creating it manually has the same effect.
My initial histo.rs file looked something like this
#pragma version(1)
#pragma rs java_package_name(java_eu_sisik_pixelproc_histo)
#pragma rs_fp_relaxed
int32_t* reds;
int32_t* greens;
int32_t* blues;
int32_t* lumas;
void RS_KERNEL histo(uchar4 in) {
rsAtomicInc(&reds[in.r]);
rsAtomicInc(&greens[in.g]);
rsAtomicInc(&blues[in.b]);
uchar l = round(0.299f*in.r + 0.587f*in.g + 0.114f*in.b);
rsAtomicInc(&lumas[l]);
}You can see that I started with a couple of #pragma statements. These will be probably present in all your scripts.
The script can contains regular C functions which you can freely call from Kotlin code. These however will be executed single-threaded.
The histo() function is a mapping kernel and therefore It's prefixed with RS_KERNEL macro. This method will have parallel execution.
The global variable pointers at the beginning are bound to Allocations in Kotlin code, so that you are able to pass data between Kotlin and RenderScript. How this is done is shown in the code below.
You can have other global variables in your code (same like in regular C code), but there are some differences related to using global variables in computing kernel functions. The variables behave like they would have 2 separate values - one value in Java and a another value in RenderScript. The main difference for me was when I tried to change the pointer (int32_t* reds) bound to Allocation, to a pre-initialized array - int32_t reds[256] = {0}. I could still read the initial value of this array in Kotlin (value == 0). But executing histo() kernel had no effect on values read in Kotlin (they stayed set to 0). Therefore, you always need to use Allocation to pass data for compute kernels between Kotlin and RenderScript.
To call our compute kernel from Kotlin, you can do the following
import android.renderscript.*
...
// Initialize a RenderScript contex
val rs = RenderScript.create(applicationContext)
// Load our RenderScript code
val script = ScriptC_histo(rs)
// Prepare input Allocation that provides Bitmap pixels
// to RenderScript compute kernel
val bitmapAllocation = Allocation.createFromBitmap(rs, bitmap)
// Create output allocations that will hold histogram values
val redAlloc = Allocation.createSized(rs, Element.I32(rs), 256)
...
// Bind output allocations to pointers in RenderScript code
script.bind_reds(redAlloc)
...
// Execute my kernel function
script.forEach_histo(bitmapAllocation)
// Copy the results from RenderScript to Kotlin
val rBins = IntArray(256) {0}
redAlloc.copyTo(rBins)
...
// Cleanup
bitmapAllocation.destroy()
script.destroy()
rs.destroy()
...As mentioned earlier, even though you are able to set and read global variables declared in your RenderScript code in Kotlin, you won't see the actual results of your processing kernel in Kotlin code. Therefore you always have to use an Allocation (and bind it in similar way as I did, in case you want to return more than one Allocation from your mapping kernel).
Additionally, forEach_histo() is executed asynchronously. Therefore you cannot expect that you'll have the results right after you called it. However, redAlloc.copyTo(rBins) is executed synchronously and that's where you can get the results out of your RenderScript kernel function.
If possible, you should try to allocate some of the RenderScript variables and your data structures that hold input data somewhere outside of your critical path of execution. Specifically, initializing the RenderScript context can be potentially time consuming.
Creating a reduction kernel is also relatively simple. Here's an example of reduction kernel for finding the maximum value in an array
...
#define INT_MIN -2147483648
#pragma rs reduce(getMax) \
initializer(initMax) \
accumulator(accMax)
static void initMax(int32_t *accum) {
*accum = INT_MIN;
}
static void accMax(int32_t *accum, int32_t val) {
*accum = max(val, *accum);
}Using it in Kotlin is then very simple
// async
val reduceResult = script.reduce_getMax(myIntArray)
// get the result synchronously
val max = reduceResult.get()In the example above I've created a custom script. Android also offers some built-in functions called interinsics, that can be used without loading a custom script. For example, this is how you could create a blur effect on you bitmap
val rs = RenderScript.create(applicationContext)
val script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs))
val inAlloc = Allocation.createFromBitmap(rs, inBitmap)
val outAlloc = Allocation.createFromBitmap(rs, outBitmap)
script.setRadius(0.5f)
script.setInput(inAlloc)
script.forEach(outAlloc)
outAlloc.copyTo(outBitmap) There is even the ScriptIntrinsicHistogram, tough I'm not sure if I can use it the same way I intended to use my histogram. I might have a look at it later.
Custom View for Showing Histogram
Once you have your histogram values, it's time to plot them on a graph.
I decided to create a simple custom view for this.
My view will display a curve for each of the colors and luma values.
As shown in official docs, I started by extending the View class
class HistogramView(context: Context, attrs: AttributeSet?): View(context, attrs) {
...
}There are multiple methods from the View class that you can override, but normally you at least want to override onMeasure() and onDraw().
Here's how I've overridden onMeasure() for my histogram View
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
var contentWidth = if (suggestedMinimumWidth > 256) suggestedMinimumWidth else 256
var contentHeight = if (suggestedMinimumHeight > 256) suggestedMinimumHeight else 256
val desiredWidth = contentWidth + paddingLeft + paddingRight
val desiredHeight = contentHeight + paddingTop + paddingBottom
val widthSpecs = resolveSizeAndState(desiredWidth, widthMeasureSpec, 0)
val heightSpecs = resolveSizeAndState(desiredHeight, heightMeasureSpec, 0)
setMeasuredDimension(widthSpecs, heightSpecs)
}When the layout hierarchy is drawn, the parent view calls measure which in turn calls onMeasure() during the measure pass. This gives your view the opportunity to give parent a hint about its required size.
The parent passes some additional info to your view through the parameters. Even though the parameters are 2 ints, there is more information stored in each of them. You can extract the information with getMode() and getSize() helper
val mode = MeasureSpec.getMode(widthMeasureSpec)
val width = MeasureSpec.getSize(widthMeasureSpec)The mode gives parent's requirements.UNSPECIFIED means your custom View can be as large as it wants. EXACT means that the parent determined exact size for it. AT_MOST gives you the maximal size, but your view can still decide to be smaller.
Important thing to note here is that this method can be called multiple times before your View is shown (for example when the parents first want to know how much all children need combined, before it puts restrictions).
In my case I used getSuggestedMinimumWidth() and getSuggestedMinimumHeight() to get some minimal size. These will take into account the minimum height set by your View, or by the background drawable that you've used for your view (whichever one is larger). In case there is no minimal height set and no background drawable that would require a minimal height, I somewhat randomly chose 256 pixels for width and height (one pixel for each histogram value).
I also took padding into account.
Then I used resolveSizeAndState() to get my actual MeasureSpec. This is a helper method which automatically returns your desired size, if parent didn't impose any constraints (or parent's constrained size if he did). This way you can avoid the if-else logic, if you don't need anything special here. It additionally packs the MeasureSpec into the returned integer.
Finally there is the requirement to call setMeasuredDimension() to store your measured values. You'll get an exception if you don't call it.
And here is finally how you can draw a curve for the red values in onDraw()
class HistogramView(context: Context, attrs: AttributeSet?): View(context, attrs) {
var range = 255
var reds: IntArray? = null
override fun onDraw(canvas: Canvas?) {
super.onDraw(canvas)
if (reds != null) {
val paint = Paint().apply {
this.color = Color.RED
style = Paint.Style.STROKE
strokeWidth = 3f
}
Path().run {
moveTo(paddingLeft.toFloat(), height - paddingBottom.toFloat())
val graphWidth = width - paddingLeft - paddingRight
val graphHeight = height - paddingTop - paddingBottom
for (i in 0 until reds!!.size) {
val x = (graphWidth * i.toFloat() / reds!!.size) + paddingLeft
val y = (graphHeight - (graphHeight * reds!![i].toFloat() / range)) + paddingBottom // flip y
lineTo(x, y)
}
canvas?.drawPath(this, paint)
}
}
}
...The intensity values for the red color were shown on the y axis. I assumed the our previously normalized range of 0 to 255, and then I just used the ratio between intensity and actual canvas height to calculate y value. Keep in mind that the coordinates are starting from top left corner, so I needed to flip the y coord.
After drawing all curves (R, G, B, and Luma), the final result looks like this
Conclusion
In this post I showed how to calculate image histogram values in Kotlin. I also showed how to access Android's Bitmap efficiently from C++ code and how to retrieve the calculated results back to Kotlin. Even though there might have been no performance improvements, I tried to implement the same calculation using RenderScript.
Then I created a simple custom view that displayed and image histogram with values plotted on a graph.
If you're interested in photography and image processing, you might find my other posts related to OpenCV and Android's camera API also useful.
Additionally, I've created a web version of image histogram generator. It can show histograms for multiple images and it also allows you to export the values into a csv file, so you might use it during your own experimentation.
Bugjaeger
Android expert tools packed into an app
Android screen mirroring/sharing tool