feat: 切换后端至PaddleOCR-NCNN,切换工程为CMake

1.项目后端整体迁移至PaddleOCR-NCNN算法,已通过基本的兼容性测试
2.工程改为使用CMake组织,后续为了更好地兼容第三方库,不再提供QMake工程
3.重整权利声明文件,重整代码工程,确保最小化侵权风险

Log: 切换后端至PaddleOCR-NCNN,切换工程为CMake
Change-Id: I4d5d2c5d37505a4a24b389b1a4c5d12f17bfa38c
This commit is contained in:
wangzhengyang
2022-05-10 09:54:44 +08:00
parent ecdd171c6f
commit 718c41634f
10018 changed files with 3593797 additions and 186748 deletions

View File

@ -0,0 +1,45 @@
Introduction to OpenCV.js and Tutorials {#tutorial_js_intro}
=======================================
OpenCV
------
OpenCV was created at Intel in 1999 by **Gary Bradski**. The first release came out in 2000. **Vadim Pisarevsky** joined Gary Bradski to manage Intel's Russian software OpenCV team. In 2005, OpenCV was used on Stanley; the vehicle that won the 2005 DARPA Grand Challenge. Later, its active development continued under the support of Willow Garage, with Gary Bradski and Vadim Pisarevsky leading the project. OpenCV now supports a multitude of algorithms related to Computer Vision and Machine Learning and is expanding day by day.
OpenCV supports a wide variety of programming languages such as C++, Python, and Java, and is available on different platforms including Windows, Linux, OS X, Android, and iOS. Interfaces for high-speed GPU operations based on CUDA and OpenCL are also under active development. OpenCV.js brings OpenCV to the open web platform and makes it available to the JavaScript programmer.
OpenCV.js: OpenCV for the JavaScript programmer
-------------
Web is the most ubiquitous open computing platform. With HTML5 standards implemented in every browser, web applications are able to render online video with HTML5 video tags, capture webcam video via WebRTC API, and access each pixel of a video frame via canvas API. With abundance of available multimedia content, web developers are in need of a wide array of image and vision processing algorithms in JavaScript to build innovative applications. This requirement is even more essential for emerging applications on the web, such as Web Virtual Reality (WebVR) and Augmented Reality (WebAR). All of these use cases demand efficient implementations of computation-intensive vision kernels on web.
[Emscripten](https://emscripten.org/) is an LLVM-to-JavaScript compiler. It takes LLVM bitcode - which can be generated from C/C++ using clang, and compiles that into asm.js or WebAssembly that can execute directly inside the web browsers. . Asm.js is a highly optimizable, low-level subset of JavaScript. Asm.js enables ahead-of-time compilation and optimization in JavaScript engine that provide near-to-native execution speed. WebAssembly is a new portable, size- and load-time-efficient binary format suitable for compilation to the web. WebAssembly aims to execute at native speed. WebAssembly is currently being designed as an open standard by W3C.
OpenCV.js is a JavaScript binding for selected subset of OpenCV functions for the web platform. It allows emerging web applications with multimedia processing to benefit from the wide variety of vision functions available in OpenCV. OpenCV.js leverages Emscripten to compile OpenCV functions into asm.js or WebAssembly targets, and provides a JavaScript APIs for web application to access them. The future versions of the library will take advantage of acceleration APIs that are available on the Web such as SIMD and multi-threaded execution.
OpenCV.js was initially created in Parallel Architectures and Systems Group at University of California Irvine (UCI) as a research project funded by Intel Corporation. OpenCV.js was further improved and integrated into the OpenCV project as part of Google Summer of Code 2017 program.
OpenCV.js Tutorials
-----------------------
OpenCV introduces a new set of tutorials that will guide you through various functions available in OpenCV.js. **This guide is mainly focused on OpenCV 3.x version**.
The purpose of OpenCV.js tutorials is to:
-# Help with adaptability of OpenCV in web development
-# Help the web community, developers and computer vision researchers to interactively access a variety of web-based OpenCV examples to help them understand specific vision algorithms.
Because OpenCV.js is able to run directly inside browser, the OpenCV.js tutorial web pages are intuitive and interactive. For example, using WebRTC API and evaluating JavaScript code would allow developers to change the parameters of CV functions and do live CV coding on web pages to see the results in real time.
Prior knowledge of JavaScript and web application development is recommended to understand this guide.
Contributors
------------
Below is the list of contributors of OpenCV.js bindings and tutorials.
- Sajjad Taheri (Architect of the initial version and GSoC mentor, University of California, Irvine)
- Congxiang Pan (GSoC student, Shanghai Jiao Tong University)
- Gang Song (GSoC student, Shanghai Jiao Tong University)
- Wenyao Gan (Student intern, Shanghai Jiao Tong University)
- Mohammad Reza Haghighat (Project initiator & sponsor, Intel Corporation)
- Ningxin Hu (Students' supervisor, Intel Corporation)

View File

@ -0,0 +1,345 @@
Using OpenCV.js In Node.js {#tutorial_js_nodejs}
==========================
Goals
-----
In this tutorial, you will learn:
- Use OpenCV.js in a [Node.js](https://nodejs.org) application.
- Load images with [jimp](https://www.npmjs.com/package/jimp) in order to use them with OpenCV.js.
- Using [jsdom](https://www.npmjs.com/package/canvas) and [node-canvas](https://www.npmjs.com/package/canvas) to support `cv.imread()`, `cv.imshow()`
- The basics of [emscripten](https://emscripten.org/) APIs, like [Module](https://emscripten.org/docs/api_reference/module.html) and [File System](https://emscripten.org/docs/api_reference/Filesystem-API.html) on which OpenCV.js is based.
- Learn Node.js basics. Although this tutorial assumes the user knows JavaScript, experience with Node.js is not required.
@note Besides giving instructions to run OpenCV.js in Node.js, another objective of this tutorial is to introduce users to the basics of [emscripten](https://emscripten.org/) APIs, like [Module](https://emscripten.org/docs/api_reference/module.html) and [File System](https://emscripten.org/docs/api_reference/Filesystem-API.html) and also Node.js.
Minimal example
-----
Create a file `example1.js` with the following content:
@code{.js}
// Define a global variable 'Module' with a method 'onRuntimeInitialized':
Module = {
onRuntimeInitialized() {
// this is our application:
console.log(cv.getBuildInformation())
}
}
// Load 'opencv.js' assigning the value to the global variable 'cv'
cv = require('./opencv.js')
@endcode
### Execute it ###
- Save the file as `example1.js`.
- Make sure the file `opencv.js` is in the same folder.
- Make sure [Node.js](https://nodejs.org) is installed on your system.
The following command should print OpenCV build information:
@code{.bash}
node example1.js
@endcode
### What just happened? ###
* **In the first statement**:, by defining a global variable named 'Module', emscripten will call `Module.onRuntimeInitialized()` when the library is ready to use. Our program is in that method and uses the global variable `cv` just like in the browser.
* The statement **"cv = require('./opencv.js')"** requires the file `opencv.js` and assign the return value to the global variable `cv`.
`require()` which is a Node.js API, is used to load modules and files.
In this case we load the file `opencv.js` form the current folder, and, as said previously emscripten will call `Module.onRuntimeInitialized()` when its ready.
* See [emscripten Module API](https://emscripten.org/docs/api_reference/module.html) for more details.
Working with images
-----
OpenCV.js doesn't support image formats so we can't load png or jpeg images directly. In the browser it uses the HTML DOM (like HTMLCanvasElement and HTMLImageElement to decode and decode images). In node.js we will need to use a library for this.
In this example we use [jimp](https://www.npmjs.com/package/jimp), which supports common image formats and is pretty easy to use.
### Example setup ###
Execute the following commands to create a new node.js package and install [jimp](https://www.npmjs.com/package/jimp) dependency:
@code{.bash}
mkdir project1
cd project1
npm init -y
npm install jimp
@endcode
### The example ###
@code{.js}
const Jimp = require('jimp');
async function onRuntimeInitialized(){
// load local image file with jimp. It supports jpg, png, bmp, tiff and gif:
var jimpSrc = await Jimp.read('./lena.jpg');
// `jimpImage.bitmap` property has the decoded ImageData that we can use to create a cv:Mat
var src = cv.matFromImageData(jimpSrc.bitmap);
// following lines is copy&paste of opencv.js dilate tutorial:
let dst = new cv.Mat();
let M = cv.Mat.ones(5, 5, cv.CV_8U);
let anchor = new cv.Point(-1, -1);
cv.dilate(src, dst, M, anchor, 1, cv.BORDER_CONSTANT, cv.morphologyDefaultBorderValue());
// Now that we are finish, we want to write `dst` to file `output.png`. For this we create a `Jimp`
// image which accepts the image data as a [`Buffer`](https://nodejs.org/docs/latest-v10.x/api/buffer.html).
// `write('output.png')` will write it to disk and Jimp infers the output format from given file name:
new Jimp({
width: dst.cols,
height: dst.rows,
data: Buffer.from(dst.data)
})
.write('output.png');
src.delete();
dst.delete();
}
// Finally, load the open.js as before. The function `onRuntimeInitialized` contains our program.
Module = {
onRuntimeInitialized
};
cv = require('./opencv.js');
@endcode
### Execute it ###
- Save the file as `exampleNodeJimp.js`.
- Make sure a sample image `lena.jpg` exists in the current directory.
The following command should generate the file `output.png`:
@code{.bash}
node exampleNodeJimp.js
@endcode
Emulating HTML DOM and canvas
-----
As you might already seen, the rest of the examples use functions like `cv.imread()`, `cv.imshow()` to read and write images. Unfortunately as mentioned they won't work on Node.js since there is no HTML DOM.
In this section, you will learn how to use [jsdom](https://www.npmjs.com/package/canvas) and [node-canvas](https://www.npmjs.com/package/canvas) to emulate the HTML DOM on Node.js so those functions work.
### Example setup ###
As before, we create a Node.js project and install the dependencies we need:
@code{.bash}
mkdir project2
cd project2
npm init -y
npm install canvas jsdom
@endcode
### The example ###
@code{.js}
const { Canvas, createCanvas, Image, ImageData, loadImage } = require('canvas');
const { JSDOM } = require('jsdom');
const { writeFileSync, existsSync, mkdirSync } = require("fs");
// This is our program. This time we use JavaScript async / await and promises to handle asynchronicity.
(async () => {
// before loading opencv.js we emulate a minimal HTML DOM. See the function declaration below.
installDOM();
await loadOpenCV();
// using node-canvas, we an image file to an object compatible with HTML DOM Image and therefore with cv.imread()
const image = await loadImage('./lena.jpg');
const src = cv.imread(image);
const dst = new cv.Mat();
const M = cv.Mat.ones(5, 5, cv.CV_8U);
const anchor = new cv.Point(-1, -1);
cv.dilate(src, dst, M, anchor, 1, cv.BORDER_CONSTANT, cv.morphologyDefaultBorderValue());
// we create an object compatible HTMLCanvasElement
const canvas = createCanvas(300, 300);
cv.imshow(canvas, dst);
writeFileSync('output.jpg', canvas.toBuffer('image/jpeg'));
src.delete();
dst.delete();
})();
// Load opencv.js just like before but using Promise instead of callbacks:
function loadOpenCV() {
return new Promise(resolve => {
global.Module = {
onRuntimeInitialized: resolve
};
global.cv = require('./opencv.js');
});
}
// Using jsdom and node-canvas we define some global variables to emulate HTML DOM.
// Although a complete emulation can be archived, here we only define those globals used
// by cv.imread() and cv.imshow().
function installDOM() {
const dom = new JSDOM();
global.document = dom.window.document;
// The rest enables DOM image and canvas and is provided by node-canvas
global.Image = Image;
global.HTMLCanvasElement = Canvas;
global.ImageData = ImageData;
global.HTMLImageElement = Image;
}
@endcode
### Execute it ###
- Save the file as `exampleNodeCanvas.js`.
- Make sure a sample image `lena.jpg` exists in the current directory.
The following command should generate the file `output.jpg`:
@code{.bash}
node exampleNodeCanvas.js
@endcode
Dealing with files
-----
In this tutorial you will learn how to configure emscripten so it uses the local filesystem for file operations instead of using memory. Also it tries to describe how [files are supported by emscripten applications](https://emscripten.org/docs/api_reference/Filesystem-API.html)
Accessing the emscripten filesystem is often needed in OpenCV applications for example to load machine learning models such as the ones used in @ref tutorial_dnn_googlenet and @ref tutorial_dnn_javascript.
### Example setup ###
Before the example, is worth consider first how files are handled in emscripten applications such as OpenCV.js. Remember that OpenCV library is written in C++ and the file opencv.js is just that C++ code being translated to JavaScript or WebAssembly by emscripten C++ compiler.
These C++ sources use standard APIs to access the filesystem and the implementation often ends up in system calls that read a file in the hard drive. Since JavaScript applications in the browser don't have access to the local filesystem, [emscripten emulates a standard filesystem](https://emscripten.org/docs/api_reference/Filesystem-API.html) so compiled C++ code works out of the box.
In the browser, this filesystem is emulated in memory while in Node.js there's also the possibility of using the local filesystem directly. This is often preferable since there's no need of copy file's content in memory. This section is explains how to do do just that, this is, configuring emscripten so files are accessed directly from our local filesystem and relative paths match files relative to the current local directory as expected.
### The example ###
The following is an adaptation of @ref tutorial_js_face_detection.
@code{.js}
const { Canvas, createCanvas, Image, ImageData, loadImage } = require('canvas');
const { JSDOM } = require('jsdom');
const { writeFileSync, readFileSync } = require('fs');
(async () => {
await loadOpenCV();
const image = await loadImage('lena.jpg');
const src = cv.imread(image);
let gray = new cv.Mat();
cv.cvtColor(src, gray, cv.COLOR_RGBA2GRAY, 0);
let faces = new cv.RectVector();
let eyes = new cv.RectVector();
let faceCascade = new cv.CascadeClassifier();
let eyeCascade = new cv.CascadeClassifier();
// Load pre-trained classifier files. Notice how we reference local files using relative paths just
// like we normally would do
faceCascade.load('./haarcascade_frontalface_default.xml');
eyeCascade.load('./haarcascade_eye.xml');
let mSize = new cv.Size(0, 0);
faceCascade.detectMultiScale(gray, faces, 1.1, 3, 0, mSize, mSize);
for (let i = 0; i < faces.size(); ++i) {
let roiGray = gray.roi(faces.get(i));
let roiSrc = src.roi(faces.get(i));
let point1 = new cv.Point(faces.get(i).x, faces.get(i).y);
let point2 = new cv.Point(faces.get(i).x + faces.get(i).width, faces.get(i).y + faces.get(i).height);
cv.rectangle(src, point1, point2, [255, 0, 0, 255]);
eyeCascade.detectMultiScale(roiGray, eyes);
for (let j = 0; j < eyes.size(); ++j) {
let point1 = new cv.Point(eyes.get(j).x, eyes.get(j).y);
let point2 = new cv.Point(eyes.get(j).x + eyes.get(j).width, eyes.get(j).y + eyes.get(j).height);
cv.rectangle(roiSrc, point1, point2, [0, 0, 255, 255]);
}
roiGray.delete();
roiSrc.delete();
}
const canvas = createCanvas(image.width, image.height);
cv.imshow(canvas, src);
writeFileSync('output3.jpg', canvas.toBuffer('image/jpeg'));
src.delete(); gray.delete(); faceCascade.delete(); eyeCascade.delete(); faces.delete(); eyes.delete()
})();
/**
* Loads opencv.js.
*
* Installs HTML Canvas emulation to support `cv.imread()` and `cv.imshow`
*
* Mounts given local folder `localRootDir` in emscripten filesystem folder `rootDir`. By default it will mount the local current directory in emscripten `/work` directory. This means that `/work/foo.txt` will be resolved to the local file `./foo.txt`
* @param {string} rootDir The directory in emscripten filesystem in which the local filesystem will be mount.
* @param {string} localRootDir The local directory to mount in emscripten filesystem.
* @returns {Promise} resolved when the library is ready to use.
*/
function loadOpenCV(rootDir = '/work', localRootDir = process.cwd()) {
if(global.Module && global.Module.onRuntimeInitialized && global.cv && global.cv.imread) {
return Promise.resolve()
}
return new Promise(resolve => {
installDOM()
global.Module = {
onRuntimeInitialized() {
// We change emscripten current work directory to 'rootDir' so relative paths are resolved
// relative to the current local folder, as expected
cv.FS.chdir(rootDir)
resolve()
},
preRun() {
// preRun() is another callback like onRuntimeInitialized() but is called just before the
// library code runs. Here we mount a local folder in emscripten filesystem and we want to
// do this before the library is executed so the filesystem is accessible from the start
const FS = global.Module.FS
// create rootDir if it doesn't exists
if(!FS.analyzePath(rootDir).exists) {
FS.mkdir(rootDir);
}
// create localRootFolder if it doesn't exists
if(!existsSync(localRootDir)) {
mkdirSync(localRootDir, { recursive: true});
}
// FS.mount() is similar to Linux/POSIX mount operation. It basically mounts an external
// filesystem with given format, in given current filesystem directory.
FS.mount(FS.filesystems.NODEFS, { root: localRootDir}, rootDir);
}
};
global.cv = require('./opencv.js')
});
}
function installDOM(){
const dom = new JSDOM();
global.document = dom.window.document;
global.Image = Image;
global.HTMLCanvasElement = Canvas;
global.ImageData = ImageData;
global.HTMLImageElement = Image;
}
@endcode
### Execute it ###
- Save the file as `exampleNodeCanvasData.js`.
- Make sure the files `aarcascade_frontalface_default.xml` and `haarcascade_eye.xml` are present in project's directory. They can be obtained from [OpenCV sources](https://github.com/opencv/opencv/tree/master/data/haarcascades).
- Make sure a sample image file `lena.jpg` exists in project's directory. It should display people's faces for this example to make sense. The following image is known to work:
![image](lena.jpg)
The following command should generate the file `output3.jpg`:
@code{.bash}
node exampleNodeCanvasData.js
@endcode

View File

@ -0,0 +1,342 @@
Build OpenCV.js {#tutorial_js_setup}
===============================
@note
You don't have to build your own copy if you simply want to start using it. Refer the Using Opencv.js tutorial for steps on getting a prebuilt copy from our releases or online documentation.
Installing Emscripten
-----------------------------
[Emscripten](https://github.com/emscripten-core/emscripten) is an LLVM-to-JavaScript compiler. We will use Emscripten to build OpenCV.js.
@note
While this describes installation of required tools from scratch, there's a section below also describing an alternative procedure to perform the same build using docker containers which is often easier.
To Install Emscripten, follow instructions of [Emscripten SDK](https://emscripten.org/docs/getting_started/downloads.html).
For example:
@code{.bash}
./emsdk update
./emsdk install latest
./emsdk activate latest
@endcode
After install, ensure the `EMSDK` environment is setup correctly.
For example:
@code{.bash}
source ./emsdk_env.sh
echo ${EMSDK}
@endcode
Modern versions of Emscripten requires to use `emcmake` / `emmake` launchers:
@code{.bash}
emcmake sh -c 'echo ${EMSCRIPTEN}'
@endcode
The version 2.0.10 of emscripten is verified for latest WebAssembly. Please check the version of Emscripten to use the newest features of WebAssembly.
For example:
@code{.bash}
./emsdk update
./emsdk install 2.0.10
./emsdk activate 2.0.10
@endcode
Obtaining OpenCV Source Code
--------------------------
You can use the latest stable OpenCV version or you can grab the latest snapshot from our [Git
repository](https://github.com/opencv/opencv.git).
### Obtaining the Latest Stable OpenCV Version
- Go to our [releases page](http://opencv.org/releases.html).
- Download the source archive and unpack it.
### Obtaining the Cutting-edge OpenCV from the Git Repository
Launch Git client and clone [OpenCV repository](http://github.com/opencv/opencv).
For example:
@code{.bash}
git clone https://github.com/opencv/opencv.git
@endcode
@note
It requires `git` installed in your development environment.
Building OpenCV.js from Source
---------------------------------------
-# To build `opencv.js`, execute python script `<opencv_src_dir>/platforms/js/build_js.py <build_dir>`.
For example, to build in `build_js` directory:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js
@endcode
@note
It requires `python` and `cmake` installed in your development environment.
-# The build script builds asm.js version by default. To build WebAssembly version, append `--build_wasm` switch.
For example, to build wasm version in `build_wasm` directory:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_wasm --build_wasm
@endcode
-# [Optional] To build the OpenCV.js loader, append `--build_loader`.
For example:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js --build_loader
@endcode
@note
The loader is implemented as a js file in the path `<opencv_js_dir>/bin/loader.js`. The loader utilizes the [WebAssembly Feature Detection](https://github.com/GoogleChromeLabs/wasm-feature-detect) to detect the features of the broswer and load corresponding OpenCV.js automatically. To use it, you need to use the UMD version of [WebAssembly Feature Detection](https://github.com/GoogleChromeLabs/wasm-feature-detect) and introduce the `loader.js` in your Web application.
Example Code:
@code{.javascipt}
// Set paths configuration
let pathsConfig = {
wasm: "../../build_wasm/opencv.js",
threads: "../../build_mt/opencv.js",
simd: "../../build_simd/opencv.js",
threadsSimd: "../../build_mtSIMD/opencv.js",
}
// Load OpenCV.js and use the pathsConfiguration and main function as the params.
loadOpenCV(pathsConfig, main);
@endcode
-# [optional] To build documents, append `--build_doc` option.
For example:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js --build_doc
@endcode
@note
It requires `doxygen` installed in your development environment.
-# [optional] To build tests, append `--build_test` option.
For example:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js --build_test
@endcode
-# [optional] To enable OpenCV contrib modules append `--cmake_option="-DOPENCV_EXTRA_MODULES_PATH=/path/to/opencv_contrib/modules/"`
For example:
@code{.bash}
python ./platforms/js/build_js.py build_js --cmake_option="-DOPENCV_EXTRA_MODULES_PATH=opencv_contrib/modules"
@endcode
-# [optional] To enable OpenCV contrib modules append `--cmake_option="-DOPENCV_EXTRA_MODULES_PATH=/path/to/opencv_contrib/modules/"`
For example:
@code{.bash}
python ./platforms/js/build_js.py build_js --cmake_option="-DOPENCV_EXTRA_MODULES_PATH=opencv_contrib/modules"
@endcode
Running OpenCV.js Tests
---------------------------------------
Remember to launch the build command passing `--build_test` as mentioned previously. This will generate test source code ready to run together with `opencv.js` file in `build_js/bin`
### Manually in your browser
To run tests, launch a local web server in `\<build_dir\>/bin` folder. For example, node http-server which serves on `localhost:8080`.
Navigate the web browser to `http://localhost:8080/tests.html`, which runs the unit tests automatically. Command example:
@code{.sh}
npx http-server build_js/bin
firefox http://localhost:8080/tests.html
@endcode
@note
This snippet and the following require [Node.js](https://nodejs.org) to be installed.
### Headless with Puppeteer
Alternatively tests can run with [GoogleChrome/puppeteer](https://github.com/GoogleChrome/puppeteer#readme) which is a version of Google Chrome that runs in the terminal (useful for Continuos integration like travis CI, etc)
@code{.sh}
cd build_js/bin
npm install
npm install --no-save puppeteer # automatically downloads Chromium package
node run_puppeteer.js
@endcode
@note
Checkout `node run_puppeteer --help` for more options to debug and reporting.
@note
The command `npm install` only needs to be executed once, since installs the tools dependencies; after that they are ready to use.
@note
Use `PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm install --no-save puppeteer` to skip automatic downloading of Chromium.
You may specify own Chromium/Chrome binary through `PUPPETEER_EXECUTABLE_PATH=$(which google-chrome)` environment variable.
**BEWARE**: Puppeteer is only guaranteed to work with the bundled Chromium, use at your own risk.
### Using Node.js.
For example:
@code{.sh}
cd build_js/bin
npm install
node tests.js
@endcode
@note If all tests are failed, then consider using Node.js from 8.x version (`lts/carbon` from `nvm`).
-# [optional] To build `opencv.js` with threads optimization, append `--threads` option.
For example:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js --build_wasm --threads
@endcode
The default threads number is the logic core number of your device. You can use `cv.parallel_pthreads_set_threads_num(number)` to set threads number by yourself and use `cv.parallel_pthreads_get_threads_num()` to get the current threads number.
@note
You should build wasm version of `opencv.js` if you want to enable this optimization. And the threads optimization only works in browser, not in node.js. You need to enable the `WebAssembly threads support` feature first with your browser. For example, if you use Chrome, please enable this flag in chrome://flags.
-# [optional] To build `opencv.js` with wasm simd optimization, append `--simd` option.
For example:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js --build_wasm --simd
@endcode
The simd optimization is experimental as wasm simd is still in development.
@note
Now only emscripten LLVM upstream backend supports wasm simd, refering to https://emscripten.org/docs/porting/simd.html. So you need to setup upstream backend environment with the following command first:
@code{.bash}
./emsdk update
./emsdk install latest-upstream
./emsdk activate latest-upstream
source ./emsdk_env.sh
@endcode
@note
You should build wasm version of `opencv.js` if you want to enable this optimization. For browser, you need to enable the `WebAssembly SIMD support` feature first. For example, if you use Chrome, please enable this flag in chrome://flags. For Node.js, you need to run script with flag `--experimental-wasm-simd`.
@note
The simd version of `opencv.js` built by latest LLVM upstream may not work with the stable browser or old version of Node.js. Please use the latest version of unstable browser or Node.js to get new features, like `Chrome Dev`.
-# [optional] To build wasm intrinsics tests, append `--build_wasm_intrin_test` option.
For example:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js --build_wasm --simd --build_wasm_intrin_test
@endcode
For wasm intrinsics tests, you can use the following function to test all the cases:
@code{.js}
cv.test_hal_intrin_all()
@endcode
And the failed cases will be logged in the JavaScript debug console.
If you only want to test single data type of wasm intrinsics, you can use the following functions:
@code{.js}
cv.test_hal_intrin_uint8()
cv.test_hal_intrin_int8()
cv.test_hal_intrin_uint16()
cv.test_hal_intrin_int16()
cv.test_hal_intrin_uint32()
cv.test_hal_intrin_int32()
cv.test_hal_intrin_uint64()
cv.test_hal_intrin_int64()
cv.test_hal_intrin_float32()
cv.test_hal_intrin_float64()
@endcode
-# [optional] To build performance tests, append `--build_perf` option.
For example:
@code{.bash}
emcmake python ./opencv/platforms/js/build_js.py build_js --build_perf
@endcode
To run performance tests, launch a local web server in \<build_dir\>/bin folder. For example, node http-server which serves on `localhost:8080`.
There are some kernels now in the performance test like `cvtColor`, `resize` and `threshold`. For example, if you want to test `threshold`, please navigate the web browser to `http://localhost:8080/perf/perf_imgproc/perf_threshold.html`. You need to input the test parameter like `(1920x1080, CV_8UC1, THRESH_BINARY)`, and then click the `Run` button to run the case. And if you don't input the parameter, it will run all the cases of this kernel.
You can also run tests using Node.js.
For example, run `threshold` with parameter `(1920x1080, CV_8UC1, THRESH_BINARY)`:
@code{.sh}
cd bin/perf
npm install
node perf_threshold.js --test_param_filter="(1920x1080, CV_8UC1, THRESH_BINARY)"
@endcode
Building OpenCV.js with Docker
---------------------------------------
Alternatively, the same build can be can be accomplished using [docker](https://www.docker.com/) containers which is often easier and more reliable, particularly in non linux systems. You only need to install [docker](https://www.docker.com/) on your system and use a popular container that provides a clean well tested environment for emscripten builds like this, that already has latest versions of all the necessary tools installed.
So, make sure [docker](https://www.docker.com/) is installed in your system and running. The following shell script should work in Linux and MacOS:
@code{.bash}
git clone https://github.com/opencv/opencv.git
cd opencv
docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) emscripten/emsdk emcmake python3 ./platforms/js/build_js.py build_js
@endcode
In Windows use the following PowerShell command:
@code{.bash}
docker run --rm --workdir /src -v "$(get-location):/src" "emscripten/emsdk" emcmake python3 ./platforms/js/build_js.py build_js
@endcode
@warning
The example uses latest version of emscripten. If the build fails you should try a version that is known to work fine which is `2.0.10` using the following command:
@code{.bash}
docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) emscripten/emsdk:2.0.10 emcmake python3 ./platforms/js/build_js.py build_js
@endcode
In Windows use the following PowerShell command:
@code{.bash}
docker run --rm --workdir /src -v "$(get-location):/src" "emscripten/emsdk:2.0.10" emcmake python3 ./platforms/js/build_js.py build_js
@endcode
### Building the documentation with Docker
To build the documentation `doxygen` needs to be installed. Create a file named `Dockerfile` with the following content:
```
FROM emscripten/emsdk:2.0.10
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends doxygen \
&& rm -rf /var/lib/apt/lists/*
```
Then we build the docker image and name it `opencv-js-doc` with the following command (that needs to be run only once):
@code{.bash}
docker build . -t opencv-js-doc
@endcode
Now run the build command again, this time using the new image and passing `--build_doc`:
@code{.bash}
docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) "opencv-js-doc" emcmake python3 ./platforms/js/build_js.py build_js --build_doc
@endcode

View File

@ -0,0 +1,18 @@
Introduction to OpenCV.js {#tutorial_js_table_of_contents_setup}
======================
- @subpage tutorial_js_intro
Introduction of OpenCV.js and Tutorials
- @subpage tutorial_js_usage
Get started with OpenCV.js
- @subpage tutorial_js_setup
Build OpenCV.js from source
- @subpage tutorial_js_nodejs
Using OpenCV.js In Node.js

View File

@ -0,0 +1,140 @@
Using OpenCV.js {#tutorial_js_usage}
===============================
Steps
-----
In this tutorial, you will learn how to include and start to use `opencv.js` inside a web page. You can get a copy of `opencv.js` from `opencv-{VERSION_NUMBER}-docs.zip` in each [release](https://github.com/opencv/opencv/releases), or simply download the prebuilt script from the online documentations at "https://docs.opencv.org/{VERSION_NUMBER}/opencv.js" (For example, [https://docs.opencv.org/3.4.0/opencv.js](https://docs.opencv.org/3.4.0/opencv.js). Use `master` if you want the latest build). You can also build your own copy by following the tutorial on Build Opencv.js.
### Create a web page
First, let's create a simple web page that is able to upload an image.
@code{.js}
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Hello OpenCV.js</title>
</head>
<body>
<h2>Hello OpenCV.js</h2>
<div>
<div class="inputoutput">
<img id="imageSrc" alt="No Image" />
<div class="caption">imageSrc <input type="file" id="fileInput" name="file" /></div>
</div>
</div>
<script type="text/javascript">
let imgElement = document.getElementById("imageSrc")
let inputElement = document.getElementById("fileInput");
inputElement.addEventListener("change", (e) => {
imgElement.src = URL.createObjectURL(e.target.files[0]);
}, false);
</script>
</body>
</html>
@endcode
To run this web page, copy the content above and save to a local index.html file. To run it, open it using your web browser.
@note It is a better practice to use a local web server to host the index.html.
### Include OpenCV.js
Set the URL of `opencv.js` to `src` attribute of \<script\> tag.
@note For this tutorial, we host `opencv.js` at same folder as index.html. You can also choose to use the URL of the prebuilt `opencv.js` in our online documentation.
Example for synchronous loading:
@code{.js}
<script src="opencv.js" type="text/javascript"></script>
@endcode
You may want to load `opencv.js` asynchronously by `async` attribute in \<script\> tag. To be notified when `opencv.js` is ready, you can register a callback to `onload` attribute.
Example for asynchronous loading
@code{.js}
<script async src="opencv.js" onload="onOpenCvReady();" type="text/javascript"></script>
@endcode
### Use OpenCV.js
Once `opencv.js` is ready, you can access OpenCV objects and functions through `cv` object.
For example, you can create a cv.Mat from an image by cv.imread.
@note Because image loading is asynchronous, you need to put cv.Mat creation inside the `onload` callback.
@code{.js}
imgElement.onload = function() {
let mat = cv.imread(imgElement);
}
@endcode
Many OpenCV functions can be used to process cv.Mat. You can refer to other tutorials, such as @ref tutorial_js_table_of_contents_imgproc, for details.
In this tutorial, we just show a cv.Mat on screen. To show a cv.Mat, you need a canvas element.
@code{.js}
<canvas id="outputCanvas"></canvas>
@endcode
You can use cv.imshow to show cv.Mat on the canvas.
@code{.js}
cv.imshow("outputCanvas", mat);
@endcode
Putting all of the steps together, the final index.html is shown below.
@code{.js}
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Hello OpenCV.js</title>
</head>
<body>
<h2>Hello OpenCV.js</h2>
<p id="status">OpenCV.js is loading...</p>
<div>
<div class="inputoutput">
<img id="imageSrc" alt="No Image" />
<div class="caption">imageSrc <input type="file" id="fileInput" name="file" /></div>
</div>
<div class="inputoutput">
<canvas id="canvasOutput" ></canvas>
<div class="caption">canvasOutput</div>
</div>
</div>
<script type="text/javascript">
let imgElement = document.getElementById('imageSrc');
let inputElement = document.getElementById('fileInput');
inputElement.addEventListener('change', (e) => {
imgElement.src = URL.createObjectURL(e.target.files[0]);
}, false);
imgElement.onload = function() {
let mat = cv.imread(imgElement);
cv.imshow('canvasOutput', mat);
mat.delete();
};
function onOpenCvReady() {
document.getElementById('status').innerHTML = 'OpenCV.js is ready.';
}
</script>
<script async src="opencv.js" onload="onOpenCvReady();" type="text/javascript"></script>
</body>
</html>
@endcode
@note You have to call delete method of cv.Mat to free memory allocated in Emscripten's heap. Please refer to [Memory management of Emscripten](https://emscripten.org/docs/porting/connecting_cpp_and_javascript/embind.html#memory-management) for details.
Try it
------
\htmlonly
<iframe src="../../js_setup_usage.html" width="100%"
onload="this.style.height=this.contentDocument.body.scrollHeight +'px';">
</iframe>
\endhtmlonly