Linking OpenCV to an iOS project (Simple)

For now, we have some basic framework for testing image processing and computer vision algorithms. Now it's time to add OpenCV to your project and add your first call to the library. You will learn how to convert UIImage to cv:: Mat, and make a call to the C++ library using Objective-C code.
Getting ready
First, you should download the OpenCV framework for iOS from the official website at http://opencv.org. In this book, we will use Version 2.4.6. You can use the iOS Simulator to work on this recipe. The source code for this recipe can be found in the Recipe03_LinkingOpenCV folder in the code bundle that accompanies this book.
How to do it...
The following are the main steps to accomplish the task:
Add the OpenCV framework to your project.
Convert image to the OpenCV format.
Process image with a simple OpenCV call.
Convert image back.
Display image as before.
Let's implement the described steps:
We continue modifying the previous project so that you can use it; otherwise create a new project with UIImageView. We'll start by adding the OpenCV framework to the Xcode project. There are two ways to do it.
You can add the framework as a resource as described in the previous recipe. This is a straightforward approach. Alternatively, the framework can be added through project properties by navigating to Project | Build Phases | Link Binary With Libraries. To open project properties you should click to the project name in the Project Navigator area.

Next, we'll include OpenCV header files to our project. To do so, we will modify the Recipe03_LinkingOpenCV-Prefix.pch precompiled header. To avoid conflicts, we will add the following code to the very beginning of the file, above all other imports:

#ifdef __cplusplus
#import <opencv2/opencv.hpp>
#endif
This is needed, because OpenCV redefines some names, for example, min/max functions. ios online course
Set the value of Compile Sources As property as Objective-C++. The property is available in the project settings and can be accessed by navigating to Project | Build Settings | Apple LLVM compiler 4.1 - Language.
To convert the images from UIImage to cv:: Mat, you can use the following functions:UIImage* MatToUIImage(const cv::Mat& image)
{
NSData *data = [NSData dataWithBytes:image.data length:image.elemSize()*image.total()];

CGColorSpaceRef colorSpace;

if (image.elemSize() == 1) {
colorSpace = CGColorSpaceCreateDeviceGray();
} else {
colorSpace = CGColorSpaceCreateDeviceRGB();
}

CGDataProviderRef provider = CGDataProviderCreateWithCFData((__bridge CFDataRef)data);

// Creating CGImage from cv::Mat
CGImageRef imageRef = CGImageCreate(image.cols, //width
image.rows, //height
8, //bits per component
8*image.elemSize(),//bits per pixel
image.step.p[0], //bytesPerRow
colorSpace, //colorspace
kCGImageAlphaNone|kCGBitmapByteOrderDefault,// bitmap info
provider, //CGDataProviderRef
NULL, //decode
false, //should interpolate
kCGRenderingIntentDefault //intent
);

// Getting UIImage from CGImage
UIImage *finalImage = [UIImage imageWithCGImage:imageRef];
CGImageRelease(imageRef);
CGDataProviderRelease(provider);
CGColorSpaceRelease(colorSpace);

return finalImage;
}

void UIImageToMat(const UIImage* image, cv::Mat& m,
bool alphaExist = false)
{
CGColorSpaceRef colorSpace = CGImageGetColorSpace(image.CGImage);
CGFloat cols = image.size.width, rows = image.size.height;
CGContextRef contextRef;
CGBitmapInfo bitmapInfo = kCGImageAlphaPremultipliedLast;
if (CGColorSpaceGetModel(colorSpace) == 0)
{
m.create(rows, cols, CV_8UC1);
//8 bits per component, 1 channel
bitmapInfo = kCGImageAlphaNone;
if (!alphaExist)
bitmapInfo = kCGImageAlphaNone;
contextRef = CGBitmapContextCreate(m.data, m.cols, m.rows, 8,
m.step[0], colorSpace,
bitmapInfo);
}
else
{
m.create(rows, cols, CV_8UC4); // 8 bits per component, 4 channels
if (!alphaExist)
bitmapInfo = kCGImageAlphaNoneSkipLast |
kCGBitmapByteOrderDefault;
contextRef = CGBitmapContextCreate(m.data, m.cols, m.rows, 8,
m.step[0], colorSpace,
bitmapInfo);
}
CGContextDrawImage(contextRef, CGRectMake(0, 0, cols, rows),
image.CGImage);
CGContextRelease(contextRef);
}

These functions are included in the library starting from Version 2.4.6 of OpenCV. To use them, you should include the ios.h header file.#import "opencv2/highgui/ios.h"
We won't explain these functions in this recipe, because it requires from readers some knowledge about CGImage and UIImage classes; but the use of these methods is really simple. Let's consider a simple example that extracts edges from the image. To do so, you have to add the following code to the viewDidLoad() method:- (void)viewDidLoad
{
[super viewDidLoad];

UIImage* image = [UIImage imageNamed:@"lena.png"];
// Convert UIImage* to cv::Mat
UIImageToMat(image, cvImage);
if (!cvImage.empty())
{
cv::Mat gray;
// Convert the image to grayscale
cv::cvtColor(cvImage, gray, CV_RGBA2GRAY);
// Apply Gaussian filter to remove small edges
cv::GaussianBlur(gray, gray,
cv::Size(5, 5), 1.2, 1.2);
// Calculate edges with Canny
cv::Mat edges;
cv::Canny(gray, edges, 0, 50);
// Fill image with white color
cvImage.setTo(cv::Scalar::all(255));
// Change color on edges
cvImage.setTo(cv::Scalar(0, 128, 255, 255), edges);
// Convert cv::Mat to UIImage* and show the resulting image
imageView.image = MatToUIImage(cvImage);
}
}
Now run your application and check whether the application finds edges on the image correctly.
How it works...
Frameworks are intended to simplify the process of handling dependencies. They encapsulate header and binary files, so the Xcode sees them, and you don't need to add all the paths manually. Simply speaking, the iOS framework is just a specially structured folder containing include files and static libraries for different architectures (for example, armv7, armv7s, and x86). But Xcode knows where to search for proper binaries for each build configuration, so this approach is the simplest way to link the external library on iOS. All dependencies are handled automatically and added to the final application package. 
Usually, iOS applications are written in Objective-C language. Header files have a *.h extension and source files have *.m. Objective-C is a superset of C, so you can easily mix these languages in one file. But OpenCV is primarily written in C++, so we need to use C++ in the iOS project, and we need to enable support of Objective-C++. That's why we have set the language property to Objective-C++. Source files in Objective-C++ language usually have the *.mm extension.
To include OpenCV header files, we use the #import directive. It is very similar to #include in C++, while there is one distinction. It automatically adds guards for the included file, while in C++ we usually add them manually:
#ifndef __SAMPLE_H__
#define __SAMPLE_H__
…
#endif
In the code of the example, we just convert the loaded image from a UIImage object to cv::Mat by calling the UIImageToMat function. Please be careful with this function, because it entails a memory copy, so frequent calls to this function will negatively affect your application's performance.

After converting images, we do some simple image processing with OpenCV. First, we convert our image to the single-channel one. After that, we use the gaussian blur filter to remove small details. Then we use the Canny method to detect edges in the image. To visualize results, we create a white image and change the color of the pixels that lie on detected edges. The resulting cv::Mat object is converted back to UIImage and displayed on the screen.
There's more...
The following is additional advice.
Objective-C++
There is one more way to add support to Objective-C++ to your project. You should just change the extension of the source files.mm where you plan to use C++ code. This extension is specific to the Objective-C++ code.
Converting to cv::Mat
If you don't want to use UIImage, but want to load an image to cv::Mat directly, you can do it using the following code:
// Create filehandle
NSFileHandle* handle =
[NSFileHandle fileHandleForReadingAtPath:filePath];
// Read content of the file
NSData* data = [handle readDataToEndOfFile];
// Decode image from the data buffer
cvImage = cv::imdecode(cv::Mat(1, [data length], CV_8UC1,
(void*)data.bytes),
CV_LOAD_IMAGE_UNCHANGED);

In this example, we read the file content to the buffer and call the cv::imdecode function to decode the image. But there is one important note; if you later want to convert cv::Mat to the UIImage, you should change the channel order from BGR to RGB, as OpenCV's native image format is BGR.