[Artoolkit] Android Sample of nftSimple

结合：[Artoolkit] ARToolKit's SDK Structure on Android

重难点：aRBaseLib/, nftSimple/, libcpufeatures.a

• aRBaseLib/

如果摄像头控制只能在android层操作，那么ARBaseLib中的摄像头控制部分就是重点之一。

unsw@unsw-UX303UB$ _cmd-wc-java
    ./AndroidUtils.java
    ./NativeInterface.java
    ./ARToolKit.java
    ./camera/CaptureCameraPreview.java
    ./camera/CameraPreferencesActivity.java
    ./camera/CameraWrapper.java
     ./camera/CameraEventListener.java
    ./rendering/gles20/ARRendererGLES20.java
     ./rendering/gles20/LineGLES20.java
     ./rendering/gles20/BaseFragmentShader.java
    ./rendering/gles20/ShaderProgram.java
     ./rendering/gles20/BaseShaderProgram.java
     ./rendering/gles20/CubeGLES20.java
     ./rendering/gles20/OpenGLShader.java
     ./rendering/gles20/ARDrawableOpenGLES20.java
    ./rendering/gles20/BaseVertexShader.java
    ./rendering/Cube.java
     ./rendering/ARRenderer.java
     ./rendering/RenderUtils.java
    ./rendering/Line.java
     ./assets/AssetFileTransferException.java
     ./assets/HashComputationException.java
    ./assets/AssetFileTransfer.java
    ./assets/Hasher.java
    ./assets/AssetHelper.java
    ./FPSCounter.java
    ./ARActivity.java
   total
unsw@unsw-UX303UB$

4000行代码，从摄像头控制开始。(Replaced with opencv4android)

01. rendering.ARActivity是main()，其中：

CameraEventListener接口可替换为opencv4android提供的CameraBridgeViewBase.CvCameraViewListener2接口。

02. 处理每一帧

@Override
    public void cameraPreviewFrame(byte[] frame) {

        if (firstUpdate) {
            // ARToolKit has been initialised. The renderer can now add markers, etc...
            if (renderer.configureARScene()) {
                Log.i(TAG, "Scene configured successfully");
            } else {
                // Error
                Log.e(TAG, "Error configuring scene. Cannot continue.");
                finish();
            }
            firstUpdate = false;
        }

        if (ARToolKit.getInstance().convertAndDetect(frame)) {

            // Update the renderer as the frame has changed
            if (glView != null) glView.requestRender();

            onFrameProcessed();
        }

    } 

03. ARToolKit处理帧，然后调用native（NDK）

    public boolean convertAndDetect(byte[] frame) {

        if (!initedNative) return false;
        if (frame == null) return false;        

        if (!NativeInterface.arwAcceptVideoImage(frame, frameWidth, frameHeight, cameraIndex, cameraIsFrontFacing)) return false;
        if (!NativeInterface.arwCapture()) return false;
        return NativeInterface.arwUpdateAR();
    }

libARWrapper.so的native代码在哪里？

nftSimple没有使用Wrapper技术，而是直接调用native: "AR and Rendering Code in Native C/C++ Using Android NDK"

怎么感觉 aRBaseLib 没有大用 ?

    @Override
    public void onPreviewFrame(byte[] data, Camera cam) {

        nftSimpleActivity.nativeVideoFrame(data);
        cam.addCallbackBuffer(data);
    }

04. 一帧的处理

 JNIEXPORT void JNICALL JNIFUNCTION_NATIVE(nativeVideoFrame(JNIEnv* env, jobject obj, jbyteArray pinArray))
 {
     int i, j, k;
     jbyte* inArray;

     if (!videoInited) {
 #ifdef DEBUG
         LOGD("nativeVideoFrame !VIDEO\n");
 #endif
         return; // No point in trying to track until video is inited.
     }
     if (!nftDataLoaded) {
         if (!nftDataLoadingThreadHandle || threadGetStatus(nftDataLoadingThreadHandle) < ) {
 #ifdef DEBUG
             LOGD("nativeVideoFrame !NFTDATA\n");
 #endif
             return;
         } else {
             nftDataLoaded = true;
             threadWaitQuit(nftDataLoadingThreadHandle);
             threadFree(&nftDataLoadingThreadHandle); // Clean up.
         }
     }
     if (!gARViewInited) {
         return; // Also, we won't track until the ARView has been inited.
 #ifdef DEBUG
         LOGD("nativeVideoFrame !ARVIEW\n");
 #endif
     }
 #ifdef DEBUG
     LOGD("nativeVideoFrame\n");
 #endif        

     // Copy the incoming  YUV420 image in pinArray.
     env->GetByteArrayRegion(pinArray, , gVideoFrameSize, (jbyte *)gVideoFrame);

     // As of ARToolKit v5.0, NV21 format video frames are handled natively,
     // and no longer require colour conversion to RGBA.
     // If you still require RGBA format information from the video,
     // here is where you'd do the conversion:
     // color_convert_common(gVideoFrame, gVideoFrame + videoWidth*videoHeight, videoWidth, videoHeight, myRGBABuffer);

     videoFrameNeedsPixelBufferDataUpload = true; // Note that buffer needs uploading. (Upload must be done on OpenGL context's thread.)

     // Run marker detection on frame
     if (trackingThreadHandle) {
         // Perform NFT tracking.
         float            err;
         int              ret;
         int              pageNo;
         // 又到了熟悉的地方：）
         if( detectedPage == - ) {
             trackingInitStart( trackingThreadHandle, gVideoFrame );
             detectedPage = -;
         }
         if( detectedPage == - ) {
             ret = trackingInitGetResult( trackingThreadHandle, trackingTrans, &pageNo);
             if( ret ==  ) {
                 if (pageNo >=  && pageNo < surfaceSetCount) {
 #ifdef DEBUG
                     LOGE("Detected page %d.\n", pageNo);
 #endif
                     detectedPage = pageNo;
                     ar2SetInitTrans(surfaceSet[detectedPage], trackingTrans);
                 } else {
                     LOGE("Detected bad page %d.\n", pageNo);
                     detectedPage = -;
                 }
             } else if( ret <  ) {
 #ifdef DEBUG
                 LOGE("No page detected.\n");
 #endif
                 detectedPage = -;
             }
         }
         if( detectedPage >=  && detectedPage < surfaceSetCount) {
             if( ar2Tracking(ar2Handle, surfaceSet[detectedPage], gVideoFrame, trackingTrans, &err) <  ) {
 #ifdef DEBUG
                 LOGE("Tracking lost.\n");
 #endif
                 detectedPage = -;
             } else {
 #ifdef DEBUG
                 LOGE("Tracked page %d (max %d).\n", detectedPage, surfaceSetCount - );
 #endif
             }
         }
     } else {
         LOGE("Error: trackingThreadHandle\n");
         detectedPage = -;
     }

     // Update markers.
 94     for (i = ; i < markersNFTCount; i++) {
         markersNFT[i].validPrev = markersNFT[i].valid;
         if (markersNFT[i].pageNo >=  && markersNFT[i].pageNo == detectedPage) {
             markersNFT[i].valid = TRUE;
             for (j = ; j < ; j++) for (k = ; k < ; k++) markersNFT[i].trans[j][k] = trackingTrans[j][k];
         }
         else markersNFT[i].valid = FALSE;
         if (markersNFT[i].valid) {

             // Filter the pose estimate.
             if (markersNFT[i].ftmi) {
                 if (arFilterTransMat(markersNFT[i].ftmi, markersNFT[i].trans, !markersNFT[i].validPrev) < ) {
                     LOGE("arFilterTransMat error with marker %d.\n", i);
                 }
             }

             if (!markersNFT[i].validPrev) {
                 // Marker has become visible, tell any dependent objects.
                 //ARMarkerAppearedNotification
             }

             // We have a new pose, so set that.
             arglCameraViewRHf(markersNFT[i].trans, markersNFT[i].pose.T, 1.0f /*VIEW_SCALEFACTOR*/);
             // Tell any dependent objects about the update.
             //ARMarkerUpdatedPoseNotification

         } else {

             if (markersNFT[i].validPrev) {
                 // Marker has ceased to be visible, tell any dependent objects.
                 //ARMarkerDisappearedNotification
             }
         }
127     }
 }

由原来的 mainloop 变为了android preview 一帧来调用，其他native部分都相同。

• libcpufeatures.a

Link: https://developer.android.com/ndk/guides/cpu-features.html

Android 官方提供的读取cpu信息的一个库。我们这里主要对应于线程相关的部分。

---

Android Studio 2.2.3直接加载Android example出现的问题：

BoiseBaked commented on Dec 21, 2016

@PN1019, this is a tough nut. Let me try summarize and hopefully that'll help you.
Why Android is problematic to target is because every Android product provider's product looks different, has different features, is using different Android OS versions and are built with different tool versions. And the Android SDK, tools, platform build tools, and build tools change frequently and radically. Then on top of all this, every app uses a different combination of compileSdkVersion, buildToolsVersion, minSdkVersion, and targetSdkVersion and requires a valid combination of Android plugin for Gradle version and Gradle product version for building. Ouch. Finally, Android implements it own Java runtime which affects the Java compiler used. So you have to stick with the "most" mainstream development environment the Android SDK developers target.
2) You have to tell me what OS your developing your Android product on, Linux, Mac OS X (12/11) or Windows 10. If you're not using one of the three, then please start using one. Furthermore, update to the latest version of one of these development platforms and keep it up-to-date.
3) For the IDE, please use Android Studio and make sure you're using the latest and greatest version of Android Studio.
4) Regarding the Java JRE an JDK. Always install the latest and greatest JRE and JDK on your development system (devbox). Understand the difference between the two: the JRE is for executing apps compiled from Java code and the JDK is for compiling Java code to classes that make up executable Java app. There is no need to keep old versions of either on your devbox. To see which version you have on your devbox, from your devbox's command line, do this:
For the Oracle Corp JDK version: javac -version
For the Oracle Corp JRE version: java -version
Gradle source in Android Studio uses the Oracle Corp JRE even though the Gradle language doesn't look like Java source. Also, Gradle uses the Oracle Corp JDK when it compiles the Android Java code to run as an app on an Android device. But Android implements it's own JRE to run its special rendition of compiled Java classes on Android devices. This means the version of Oracle Corp JDK used by Gradle when compiling Java for Android devices will lag behind Oracle's latest JDK release. Therefore, you can't use the latest and greatest Java language features when developing an Android app because Android is using an old version of javac, i.e. an old Oracle Corp JDK version.

So there's two ways to get around this.

Use the JDK that is embedded with Android Studio 2.2.+. Or, tell Android Studio to use the Oracle Corp JDK installed on your devbox but have it run in such a way to regress in compiling an older version of the Java language creating an older version of runtime Java classes. I recommend the former, please see my entry above now labeled with "Recommended JDK to use:".

and Android Studio 2.1.1 is better.

unsw@unsw-UX303UB$ sudo update-alternatives --config javac
There are  choices for the alternative javac (providing /usr/bin/javac).

  Selection    Path                                         Priority   Status
------------------------------------------------------------
              /usr/lib/jvm/java--oracle/bin/javac                auto mode
*             /usr/lib/jvm/java--oracle/bin/javac                manual mode
              /usr/lib/jvm/java--openjdk-amd64/bin/javac         manual mode
              /usr/lib/jvm/java--oracle/bin/javac                manual mode

Press enter to keep the current choice[*], or type selection number: 

unsw@unsw-UX303UB$ sudo update-alternatives --config java
There are  choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
              /usr/lib/jvm/java--oracle/jre/bin/java                auto mode
*             /usr/lib/jvm/java--openjdk-amd64/jre/bin/java         manual mode
              /usr/lib/jvm/java--oracle/jre/bin/java                manual mode
              /usr/lib/jvm/java--openjdk-amd64/jre/bin/java         manual mode
              /usr/lib/jvm/java--oracle/jre/bin/java                manual mode

Press enter to keep the current choice[*], or type selection number: 

unsw@unsw-UX303UB$ javac -version
javac 1.7.0_80

unsw@unsw-UX303UB$ java -version
java version "1.7.0_121"
OpenJDK Runtime Environment (IcedTea 2.6.) (7u121-2.6.-1ubuntu0.14.04.)
OpenJDK -Bit Server VM (build 24.121-b00, mixed mode)

ubuntu14.14, jdk 1.7.x

Next: how to implement multi-nft，start with linux verison.