What is a C++ pipeline ?

A C++ pipeline is a shared library embedding an implementation of the pipeline interface defined by the SolAR framework (IMappingPipeline, IMapUpdatePipeline, IPoseEstimation Pipeline, IRelocalizationPipeline, etc.). Thus, this pipeline can be loaded at run-time and run by any third party application such as a C++ application, a service or by Unity (Unity supports only IPoseEstimationPipeline at this time). To provide configuration capabilities to a pipeline, it is considered by SolAR as a component embedded in a module. For this reason, readers are encouraged to have a look at the create a module section.

New pipeline interfaces will be soon added for other vision task such as 3D dense mapping, object recognition, etc.

Initialize your pipeline step-by-step

Create your pipeline project

If not already done, start by installing this wizard by launching the install.bat on Windows or install.sh on Linux (in your ${XPCF_MODULE_ROOT}/xpcf/[version]/wizards/qtcreator).

  • QT Creator

Open QTCreator and create a new project (in File menu).

App wizard QT

Then, set the name of your pipeline (e.g. MyPipeline) and the location of the QT project.

App wizard QT

Set the name of your package (you can reuse the name of your pipeline), the package version, and select the value install recursively all dependencies and check the QTVS box if you want to load your project in Visual Studio.

App wizard QT

Then, select the XPCF package path, and set a namespace for your pipeline (we recommend to set the namespace to SolAR::PIPELINES).

App wizard QT

For the next step, choose your development kits. We recommend to use MSVC 2017 64bit on Windows or Clang on Linux.

Your project is now created and the result is a .pro file like following:

MyPipeline.pro
QT       -= core gui
CONFIG -= app_bundle qt

TARGET = MyPipeline
FRAMEWORK = $${TARGET}
VERSION=0.11.0
DEFINES +=  $${TARGET}VERSION=\"$${VERSION}\"

# Uncomment following line to add more verbose information from builddefs-qmake rules
# CONFIG += verbose
# Uncomment following line to prepare remaken package
# CONFIG += package_remaken

CONFIG += with_qtvs

with_qtvs {
    PROJECTCONFIG = QTVS
}

CONFIG += c++1z
CONFIG += shared

staticlib {
    DEPENDENCIESCONFIG = staticlib
    REMAKEN_PKGSUBDIR=static
} else {
    DEPENDENCIESCONFIG = sharedlib
    REMAKEN_PKGSUBDIR=shared
}

CONFIG(debug,debug|release) {
    DEFINES += _DEBUG=1
    DEFINES += DEBUG=1
    REMAKEN_PKGSUBDIR=$${REMAKEN_PKGSUBDIR}/debug
}

CONFIG(release,debug|release) {
    DEFINES += NDEBUG=1
    REMAKEN_PKGSUBDIR=$${REMAKEN_PKGSUBDIR}/release
}

package_remaken {
    message("Preparing remaken package installation in $${REMAKEN_PKGSUBDIR}")
    INSTALLSUBDIR=$${REMAKEN_PKGSUBDIR}
}

include(findremakenrules.pri) (1)

DEPENDENCIESCONFIG = sharedlib
DEPENDENCIESCONFIG += install_recurse (2)

## Configuration for Visual Studio to install binaries and dependencies. Work also for QT Creator by replacing QMAKE_INSTALL
PROJECTCONFIG = QTVS (3)

#NOTE : CONFIG as staticlib or sharedlib, DEPENDENCIESCONFIG as staticlib or sharedlib and PROJECTDEPLOYDIR MUST BE DEFINED BEFORE templatelibbundle.pri inclusion
include ($${QMAKE_REMAKEN_RULES_ROOT}/templatelibconfig.pri)


DEFINES += BOOST_ALL_NO_LIB
DEFINES += BOOST_ALL_DYN_LINK

SOURCES +=     MyPipeline_main.cpp

HEADERS +=     MyPipelineAPI.h
unix {
}

macx {
    DEFINES += _MACOS_TARGET_
    QMAKE_MAC_SDK= macosx
    QMAKE_CFLAGS += -mmacosx-version-min=10.7 #-x objective-c++
    QMAKE_CXXFLAGS += -mmacosx-version-min=10.7 -std=c++17 -fPIC#-x objective-c++
    QMAKE_LFLAGS += -mmacosx-version-min=10.7 -v -lstdc++
    LIBS += -lstdc++ -lc -lpthread
}

win32 {
    DEFINES += _X86_VC12_TARGET_
    DEFINES += MBCS _MBCS
 }

INCLUDEPATH += $${PWD}

header_files.path = $${PROJECTDEPLOYDIR}/interfaces/
header_files.files = $$files($${PWD}/I*.h)

INSTALLS += header_files
DISTFILES +=     Makefile

OTHER_FILES +=     packagedependencies.txt

#NOTE : Must be placed at the end of the .pro
    include ($${QMAKE_REMAKEN_RULES_ROOT}/remaken_install_target.pri) (4)
1 This .pri file has been installed by the wizard. It will allow to find the remaken folder depending on the OS you are using.
2 The dependencies of your pipeline will be installed recursively. More details are available on the builddefs-qmake project on GitHub.
3 The build and installation of your pipeline will also work with Visual Studio. Warning, in QTCreator, this will replace the usual QMAKE_INSTALL.
4 Place at the end the .pri file to install your pipeline.

Visual Studio

You can also simply create your pipeline with Visual Studio by using the .pro file (see above for more details how to configure the .pro file).

Microsoft Visual Studio provides a Qt Visual Studio Tools. This enables developers to import QT project files (.pro) into Visual Studio.

Install QT Visual Studio Tools:

  • In Visual Studio, select Tools > Extensions and Updates > Online to install and update QT Visual Studio Tools.

Import the .pro. file into Visual Studio:

  • Select Qt VS Tools > Open Qt Project File (.pro) and choose your .pro file.

Right now, your project is configured.

Select your dependencies

As mentionned previously, SolAR framework provides developers with a building pipeline allowing among other things to easily manage dependencies (download, version management, packaging during deployment step, etc.).

To define the dependencies used by your pipeline, just replace in your packagedependencies.txt the reference to XPCF package by a reference to the SolARFramework as shown below:

/.packagedependencies.txt

SolARFramework|0.11.0|SolARFramework|SolARBuild@github|https://github.com/SolarFramework/SolarFramework/releases/download

Here is the syntax for each dependency (more information available on the Remaken project on GitHub):

framework#channel | version | [condition]#library name | identifier or local folder@repository_type | repository_url | link_mode | options

As the component manager provided by XPCF can load at runtime the modules used by your pipeline as defined into your configuration file presented next, you do not need to add them into the packagedependencies.txt file. Also, XPCF and the build pipeline handle dependency recursivity, meaning that you do not need to add the dependency to XPCF already referenced by the SolAR framework package.

Your dependencies, also called artifacts, should be available in your ${XPCF_MODULE_ROOT}/ folder. To automatically download your dependencies, just run remaken install where your packagedependencies.txt is located.

Refer to https://github.com/b-com-software-basis/remaken for more information.

Numerous samples of packagedependencies.txt files can be found with the SolAR Samples you have surely installed on your machine.

Create the configuration file

This file will be used at run-time. It will allow XPCF to load Module components and configure them at run-time. So you can experiment different implementations and configurations of a pipeline without the need to recompile your application.

Creating a configuration file is very easy. Once you have identified the modules and the components required to assemble your pipeline, just record them in the xml configuration file.

Following, an example of a configuration file for a pipeline using the camera and image viewer components embedded in the SolARModuleOpenCV module :

MyPipelineConfiguration.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<xpcf-registry autoAlias="true">

<module uuid="15e1990b-86b2-445c-8194-0cbe80ede970" name="SolARModuleOpenCV" path="$XPCF_MODULE_ROOT/SolARBuild/SolARModuleOpenCV/0.11.0/lib/x86_64/shared" description="OpenCV"> (1)
  <component uuid="5B7396F4-A804-4F3C-A0EB-FB1D56042BB4" name="SolARCameraOpencv" description="SolARCameraOpencv"> (2)
    <interface uuid="125f2007-1bf9-421d-9367-fbdc1210d006" name="IComponentIntrospect" description="IComponentIntrospect"/>
    <interface uuid="5DDC7DF0-8377-437F-9C81-3643F7676A5B" name="ICamera" description="ICamera"/>
  </component>
  <component uuid="fa4a780a-9720-11e8-9eb6-529269fb1459" name="SolARVideoAsCameraOpencv" description="SolARVideoAsCameraOpencv">
      <interface uuid="125f2007-1bf9-421d-9367-fbdc1210d006" name="IComponentIntrospect" description="IComponentIntrospect"/>
      <interface uuid="5DDC7DF0-8377-437F-9C81-3643F7676A5B" name="ICamera" description="ICamera"/>
  </component>
  <component uuid="19ea4e13-7085-4e3f-92ca-93f200ffb01b" name="SolARImageViewerOpencv" description="SolARImageViewerOpencv">
    <interface uuid="125f2007-1bf9-421d-9367-fbdc1210d006" name="IComponentIntrospect" description="IComponentIntrospect"/>
    <interface uuid="b05f3dbb-f93d-465c-aee1-fb58e1480c42" name="IImageViewer" description="IImageViewer"/>
  </component>
</module>
<factory>
    <bindings>
      <bind interface="ICamera" to="SolARCameraOpencv" properties="CameraProperty"/> (3)
      <bind interface="ICamera" to="SolARVideoAsCameraOpencv" name="VideoAsCamera" properties="VideoAsCameraProperty"/> (4)
    </bindings>
</factory>

<properties> (5)
  <configuration component="SolARCameraOpencv" name="CameraProperty">
    <property name="calibrationFile" type="string" value="camera_calibration.json"/>
    <property name="deviceID" type="uint" value="0"/>
  </configuration>
  <configuration component="SolARVideoAsCameraOpencv" name="VideoAsCameraProperty"> (6)
    <property name="calibrationFile" type="string" value="camera_calibration.json"/>
    <property name="videoPath" type="string" value="path to video"/>
    <property name="delayTime" type="int" value="30"/>
  </configuration>
  <configuration component="SolARImageViewerOpencv">
    <property name="title" type="string" value="Original Image"/>
    <property name="exitKey" type="int" value="27"/>
    <property name="width" type="int" value="0"/>
    <property name="height" type="int" value="0"/>
  </configuration>
</properties>

</xpcf-registry>
1 Add the modules used by your pipeline. To get information concerning the modules, have a look to their registry file in the ${HOME_DIR}/.xpcf/SolAR/.
2 For each module, add the components used by your pipeline. To get information concerning the components, have a look to the registry file of their module available in the ${HOME_DIR}/.xpcf/SolAR/.
3 Thanks to the factory field, you can bind by default a specific component implementation (here a camera openCV) on a given SolAR interface (here the ICamera).
4 You can give a name to a binding. In this case, in your code you will be able to ask for this specific binding when you will want to instantiate a ICamera component.
5 Optionally, add a configuration to your components. The name of your configuration parameters are generally the name of the variable in the class definition of the component without the prefix "m_". The type of the configuration parameters are simple types such as "String", "Integer", "Float", "Double", or array of these simple types.
6 A second configuration is defined for the SolARVideoAsCameraOpencv with a specific name if you want to instanciate a second component with a specific configuration

Examples of such configuration files are available in the samples directory that is provided with the SolAR installation.

Implement your pipeline

Using QT Creator Wizard

You can create a pipeline like a component using Qt creator wizard. This component has to inherit of one of the pipeline interfaces (have a look to SolAR Framework API in SolAR::api::pipeline).

To create a pipeline inheriting from IPoseEstimation Pipeline, follow below instructions.

Open QTCreator if not already done, and load the project of your pipeline. Be sure that your project MyPipeline is active.

Then, select in the QTCreator menu File, New File or Project…​. Select in the Files and Classes menu on the left XPCF, and then select `XPCF Component Class and Choose…​.

App wizard QT

Then, set the name of your pipeline (e.g. MyPipeline) and specify a custom base class, here SolAR::api::IPoseEstimationPipeline. If you want your pipeline to be configurable, check the corresponding box.

App wizard QT

Finally, click on Finish.

For more information, refer to create a component to create your pipeline.

Pipeline API

Some abstract classes named I???Pipeline have been defined in SolAR framework and are defined in api/pipeline/I???Pipeline.h .
Any SolAR pipeline should implement one of these classes, that means that all methods defined in these pipelines have to be implemented in the pipeline implementation. For example, for the IPoseEstimationPipeline:

  • init: Allows to execute some processes before starting the pipeline.

  • start: Starts the pipeline with a texture buffer that will be updated when a new frame will be processed and ready for display.

  • getCameraParameters: Provide third party applications with the calibration parameters of the camera. Useful when you want to set the parameters of the virtual camera with the ones from the real one.

  • update: A method that provides the new pose of the camera.

  • loadSourceImage: If there is no camera in your pipeline, you can feed it with an external image (for instance, an image capture by the Unity web camera).

A full description of these methods is available in the api/pipeline/IPoseEstimationPipeline.h file.

Pipeline Template

To implement your pipeline, you have to fill in the header file MyPipeline.h and the source file MyPipeline.cpp. Next, we provide you with templates for these two files.

MyPipeline.h
/**
 * Information concerning the copyright and licence of your pipeline
 */

#ifndef MYPIPELINE_H
#define MYPIPELINE_H

// Mandatory include file (1)
#include "MyPipelineAPI.h"
#include "xpcf/core/traits.h"
#include "xpcf/component/ConfigurableBase.h"
#include "api/pipeline/IPoseEstimationPipeline.h"
#include "xpcf/threading/BaseTask.h"

// Add here the header file for the component interfaces required by your pipeline (2)
// e.g. #include "api/input/devices/ICamera.h"

namespace xpcf=org::bcom::xpcf;

namespace SolAR { (3)
using namespace datastructure;
using namespace api;
namespace PIPELINES {

/**
 * @class MyPipeline
 * @brief A short description of your pipeline
 */
(4)
class MyPipeline_EXPORT_API MyPipeline : public org::bcom::xpcf::ConfigurableBase,
    public api::pipeline::IPoseEstimationPipeline
{
public:
    MyPipeline();
    ~MyPipeline();

    /// @brief Initialization of the pipeline
    /// @return FrameworkReturnCode::_SUCCESS if the init succeeds, else FrameworkReturnCode::_ERROR_
    FrameworkReturnCode init() override;

    /// @brief Provide the camera parameters
    /// @return the camera parameters (its resolution and its focal)
    CameraParameters getCameraParameters() const override;

    /// @brief Starts the pipeline.
    /// @return FrameworkReturnCode::_SUCCESS if the start succeeds, else FrameworkReturnCode::_ERROR_
    FrameworkReturnCode start() override;

    /// @brief Starts the pipeline and provides a texture buffer which will be updated when required.
    /// @param[in] textureHandle a pointer to the texture buffer which will be updated at each call of the update method.
    /// @return FrameworkReturnCode::_SUCCESS if the start succeeds, else FrameworkReturnCode::_ERROR_
    FrameworkReturnCode start(void* imageDataBuffer) override;

    /// @brief Stop the pipeline.
    FrameworkReturnCode stop() override;

    /// @brief update the pipeline
    /// Get the new pose and update the texture buffer with the image that has to be displayed
    api::sink::SinkReturnCode update(Transform3Df& pose) override;

    /// @brief load the source image
    api::source::SourceReturnCode loadSourceImage(void* sourceTextureHandle, int width, int height) override;

    xpcf::XPCFErrorCode onConfigured() override;

    void onInjected() override;

    void unloadComponent () override final;

private:
  // Add here the declaration of the shared references on components required by your pipeline (e.g. SRef<input::device::ICamera> m_camera;)
  (5)

  // Threads (6)
  bool pipelineLoop();
  xpcf::DelegateTask* m_pipelineLoopTask = nullptr;

  // optionally, add here some parameters of your module that can be configured
  // int m_myPipelineParameter; (7)
};

}
}


XPCF_DEFINE_COMPONENT_TRAITS(SolAR::PIPELINES::MyPipeline, (8)
                             "<yourUUID>",
                             "My pipeline",
                             "A sample pipeline used for documentation");

#endif // MYPIPELINE_H
1 These include files are mandatory. They correspond to the XPCF traits file as here we declare the module traits directly in this file, to the XPCF configurableBase header file if you want to configure your pipeline through an external xml file, and to the SolAR IPipeline header file, as your pipeline implement this SolAR interface.
2 The header files of the SolAR component interfaces required by your pipeline.
3 SolAR and XPCF namespaces directives to shorten the calls to SolAR api and data structures.
4 Declare your pipeline class, and add the functions defined as abstract in the I????Pipeline interface.
5 Declaration of the components used by the pipeline (thanks to their abstract interfaces). We are using a shared reference of the components to ease their use in multi-threaded pipelines.
6 Declare a XPCF task and the function running the pipeline loop in a dedicated thread.
7 You can declare parameters for the pipeline. Some of these parameters may be configurable.
8 Finally, add the trait of your pipeline. To generate an UUID, you can use a online UUID generator
MyPipeline.cpp
/**
 * Information concerning the copyright and licence of your pipeline
 */

// Mandatory header file (1)
#include "xpcf/module/ModuleFactory.h"
#include "core/Log.h"

// Header to your pipeline declaration (2)
#include "MyPipeline.h"

// Header to datastructure required to exchange data between your components (3)
// e.g. #include "datastructure/Image.h"

// Macro used by the XPCF factory to define a component/pipeline (4)
XPCF_DEFINE_FACTORY_CREATE_INSTANCE(SolAR::PIPELINES::MyPipeline)

namespace xpcf=org::bcom::xpcf;

namespace SolAR { (5)
using namespace datastructure;
using namespace api::pipeline;
using namespace api::source;
using namespace api::sink;
namespace PIPELINES {

MyPipeline::MyPipeline():ConfigurableBase(xpcf::toUUID<MyPipeline>())
{
   addInterface<api::pipeline::IPipeline>(this);
   // Add here the declaration of the component implementation you want to inject in your interfaces
   // e.g. declareInjectable<input::device::ICamera>(m_camera);
   // If a sepcific named binding is needed
   // e.g. declareInjectable<input::device::ICamera>(m_camera, "VideoAsCamera");
   (6)
   // Add here the mapping of your pipeline properties with the variables of your pipeline class
   // e.g. declareProperty("myPipelineProperty", m_MyPipelineVariable);
   (7)
}

MyPipeline::~MyPipeline()
{

}

void MyPipeline::onInjected()
{
    // ADD HERE: Things to do when injectable components have been inject in your pipline.
}

xpcf::XPCFErrorCode MyPipeline::onConfigured()
{
  // ADD HERE: Things to do when the variables of your pipeline class have just been set according to the properties defined in your configuration file.
}

FrameworkReturnCode MyPipeline::init()
{
  try {
    // ADD HERE: Thinks to do to initialize your pipeline
    //e.g. load a marker or set camera parameters for components requiring them.
    (8)

  }
  catch (xpcf::Exception e)
  {
     LOG_WARNING("One or more components cannot be created: {}", e.what());
     return FrameworkReturnCode::_ERROR_;
  }

  return FrameworkReturnCode::_SUCCESS;
}

CameraParameters MyPipeline::getCameraParameters() const
{
  CameraParameters camParam;
  // ADD HERE the code to return camera parameters

  return camParam;
}

bool MyPipeline::pipelineLoop() (9)
{
  // ADD HERE: the code connecting the component of your pipeline

  return true;
}

SourceReturnCode MyPipeline::loadSourceImage(void* sourceTextureHandle, int width, int height)
{
   // ADD HERE the code to take external image as input of your pipeline

   return SourceReturnCode::_NOT_IMPLEMENTED;
}


FrameworkReturnCode MyPipeline::start(void* imageDataBuffer)
{
  // ADD HERE the code to start your pipeline
  // e.g. start the camera

  // create and start a thread for the loop of the pipeline (10)
  auto pipelineLoopThread = [this](){;pipelineLoop();};
  m_pipelineLoopTask = new xpcf::DelegateTask(pipelineLoopThread);
  m_pipelineLoopTask->start();

  return FrameworkReturnCode::_SUCCESS;
}

FrameworkReturnCode MyPipeline::stop()
{
  // ADD HERE the code to stop your pipeline
  if (m_pipelineLoopTask != nullptr)
    m_pipelineLoopTask->stop(); (11)
  return FrameworkReturnCode::_SUCCESS;
}

SinkReturnCode MyPipeline::update(Transform3Df& pose)
{
  // ADD HERE the code to update the pose of the camera

  return SinkReturnCode::_NOT_IMPLEMENTED;
}

}
}
1 Required header file corresponding to the XPCF module factory.
2 Obviously, add the header file corresponding to your pipeline declaration.
3 Add the header of datastructures required to exchange information between the components of your pipeline.
4 Add the definition of your pipeline to the XPCF component factory.
5 Embed your pipeline in the SolAR/PIPELINES namespace, and add use namespace used in your pipeline.
6 If you have declare the component interfaces required for your pipeline in its header file, you can map component implementatations on them. The choice of the component implementations to map on your interfaces will be solved at runtime according to what is described in your xml configuration file. By default, the resolution chooses the first component alias declared in the configuration file that fits your interface. If you have several components alias declared in your configuration file, you can force the choice by defining a specific binding. In your pipeline, if you need several instances of a component based on different implementations, you can name your binding and pass it to the DeclareInjectable function. Do not forget to add _autoalias=true" at the beginning of your xml configuration file if you want to automatically create aliases for all declared components based on their names.
7 To set a variable of the pipeline class as configurable and map on it a property defined in the xml configuration file, you have to declare the property. Here, if an xml element with the name "myPipelineProperty" is specified in the configuration file, when exiting the constructor, its value will be set to the variable m_myPipelineVariable.
8 If required, add the code to initialize your pipeline (e.g. load a reference image, initialize some component with the intrinsic parameters of the camera, etc.).
9 Implement the loop of the pipeline by connecting components. This loop will run in a dedicated thread.
10 Start the pipeline loop thread when the start method is called.
11 Stop the pipeline loop thread when the stop method is called.

Pipeline sample

We will present next the implementation of the simplest pipeline that consists in capturing an image from a camera and make it available for a third party application. For this implementation, we will need only one module: SolARModuleOpenCV.

MyPipeline.h
/**
 * Information concerning the copyright and licence of your pipeline
 */

#ifndef MYPIPELINE_H
#define MYPIPELINE_H

// Mandatory include file
#include "MyPipelineAPI.h"
#include "xpcf/core/traits.h"
#include "xpcf/component/ConfigurableBase.h"
#include "api/pipeline/IPoseEstimationPipeline.h"
#include "xpcf/threading/BaseTask.h"

// Add here the header file for the component interfaces required by your pipeline
// e.g. #include "api/input/devices/ICamera.h" (1)
#include "api/input/devices/ICamera.h"
#include "api/sink/ISinkPoseImage.h"
#include "api/source/ISourceImage.h"

namespace xpcf=org::bcom::xpcf;

namespace SolAR {
using namespace datastructure;
using namespace api;
namespace PIPELINES {

/**
 * @class MyPipeline
 * @brief A short description of your pipeline
 */
class MyPipeline_EXPORT_API MyPipeline : public org::bcom::xpcf::ConfigurableBase,
    public api::pipeline::IPoseEstimationPipeline
{
public:
    MyPipeline();
    ~MyPipeline();

    /// @brief Initialization of the pipeline
    /// @return FrameworkReturnCode::_SUCCESS if the init succeeds, else FrameworkReturnCode::_ERROR_
    FrameworkReturnCode init() override;

    /// @brief Provide the camera parameters
    /// @return the camera parameters (its resolution and its focal)
    CameraParameters getCameraParameters() const override;

    /// @brief Starts the pipeline.
    /// @return FrameworkReturnCode::_SUCCESS if the start succeeds, else FrameworkReturnCode::_ERROR_
    FrameworkReturnCode start() override;

    /// @brief Starts the pipeline and provides a texture buffer which will be updated when required.
    /// @param[in] textureHandle a pointer to the texture buffer which will be updated at each call of the update method.
    /// @return FrameworkReturnCode::_SUCCESS if the start succeeds, else FrameworkReturnCode::_ERROR_
    FrameworkReturnCode start(void* imageDataBuffer) override;

    /// @brief Stop the pipeline.
    FrameworkReturnCode stop() override;

    /// @brief update the pipeline
    /// Get the new pose and update the texture buffer with the image that has to be displayed
    api::sink::SinkReturnCode update(Transform3Df& pose) override;

    /// @brief load the source image
    api::source::SourceReturnCode loadSourceImage(void* sourceTextureHandle, int width, int height) override;

    xpcf::XPCFErrorCode onConfigured() override;

    void onInjected() override;

    void unloadComponent () override final;

private:
  // Add here the declaration of the shared references on components required by your pipeline (e.g. SRef<input::device::ICamera> m_camera;)
  SRef<input::devices::ICamera> m_camera; (2)
  SRef<sink::ISinkPoseImage> m_sink;
  SRef<source::ISourceImage> m_source;

  // Threads
  bool pipelineLoop();
  xpcf::DelegateTask* m_pipelineLoopTask = nullptr;

  // optionally, add here some parameters of your module that can be configured
  // int m_myPipelineParameter;

  // Other attributes
  bool m_externalInputImageMode = false; (3)
};

}
}


XPCF_DEFINE_COMPONENT_TRAITS(SolAR::PIPELINES::MyPipeline,
                             "855c83b7-f4ec-48ab-8e89-56018ea9e169",
                             "My pipeline",
                             "A sample pipeline used for documentation");

#endif // MYPIPELINE_H
1 include the component interface headers for a camera component, a sink component and a source component. A sink component handles an output buffer feed by the pipeline a read by external third parties. Reciprocally, a source component handles an input buffer feeds by external third parties and used by the pipeline.
2 Declaration of the three components used by this pipeline, a camera, a sink component handling the pose and the output image, and a source component handling an input image.
3 Add an attribute to know if the pipeline use as input an image coming from a third party.
MyPipeline.cpp
/**
 * Information concerning the copyright and licence of your pipeline
 */

// Mandatory header file
#include "xpcf/module/ModuleFactory.h"
#include "core/Log.h"

// Header to your pipeline declaration
#include "MyPipeline.h"

// Header to datastructure required to exchange data between your components
// e.g. #include "datastructure/Image.h"
#include "datastructure/Image.h" (1)

// Macro used by the XPCF factory to define a component/pipeline
XPCF_DEFINE_FACTORY_CREATE_INSTANCE(SolAR::PIPELINES::MyPipeline)

namespace xpcf=org::bcom::xpcf;

namespace SolAR {
using namespace datastructure;
using namespace api::pipeline;
using namespace api::source;
using namespace api::sink;
namespace PIPELINES {

MyPipeline::MyPipeline():ConfigurableBase(xpcf::toUUID<MyPipeline>())
{
   addInterface<api::pipeline::IPipeline>(this);
   // Add here the declaration of the component implementation you want to inject in your interfaces
   // e.g. declareInjectable<input::device::ICamera>(m_camera);
   // If a sepcific named binding is needed
   // e.g. declareInjectable<input::device::ICamera>(m_camera, "VideoAsCamera");
   (2)
   declareInjectable<input::devices::ICamera>(m_camera);
   declareInjectable<sink::ISinkPoseImage>(m_sink);
   declareInjectable<source::ISourceImage>(m_source);
   // Add here the mapping of your pipeline properties with the variables of your pipeline class
   // e.g. declareProperty("myPipelineProperty", m_MyPipelineVariable);
}

MyPipeline::~MyPipeline()
{

}

void MyPipeline::onInjected()
{
    // ADD HERE: Things to do when injectable components have been inject in your pipline.
}

xpcf::XPCFErrorCode MyPipeline::onConfigured()
{
  // ADD HERE: Things to do when the variables of your pipeline class have just been set according to the properties defined in your configuration file.
}

FrameworkReturnCode MyPipeline::init()
{
  try {
    // ADD HERE: Thinks to do to initialize your pipeline
    //e.g. load a marker or set camera parameters for components requiring them.

  }
  catch (xpcf::Exception e)
  {
     LOG_WARNING("One or more components cannot be created: {}", e.what());
     return FrameworkReturnCode::_ERROR_;
  }

  return FrameworkReturnCode::_SUCCESS;
}

CameraParameters MyPipeline::getCameraParameters() const
{
  CameraParameters camParam;
  // ADD HERE the code to return camera parameters
  if (m_camera) (3)
  {
      camParam = m_camera->getParameters();
  }
  return camParam;
}

bool MyPipeline::pipelineLoop() (4)
{
  // ADD HERE: the code connecting the component of your pipeline
  SRef<Image> image;

  if (m_externalInputImageMode)
  {
    if (m_source->getNextImage(image) == SourceReturnCode::_NEW_IMAGE)
      m_sink->set(image);
  }
  else
  {
    if (m_camera->getNextImage(image) == SolAR::FrameworkReturnCode::_ERROR_LOAD_IMAGE)
      return false;
    m_sink->set(image);
  }
  return true;
}

SourceReturnCode MyPipeline::loadSourceImage(void* sourceTextureHandle, int width, int height)
{
   // ADD HERE the code to take external image as input of your pipeline
   m_externalInputImageMode = true; (5)
   return m_source->setInputTexture((unsigned char *)sourceTextureHandle, width, height);
}


FrameworkReturnCode MyPipeline::start(void* imageDataBuffer)
{
  // ADD HERE the code to start your pipeline
  // e.g. start the camera
  m_sink->setImageBuffer((unsigned char*)imageDataBuffer); (6)

  if (m_camera->start() != FrameworkReturnCode::_SUCCESS) (7)
    return FrameworkReturnCode::_ERROR_;

  // create and start a thread for the loop of the pipeline
  auto pipelineLoopThread = [this](){;pipelineLoop();};
  m_pipelineLoopTask = new xpcf::DelegateTask(pipelineLoopThread);
  m_pipelineLoopTask->start();

  return FrameworkReturnCode::_SUCCESS;
}

FrameworkReturnCode MyPipeline::stop()
{
  // ADD HERE the code to stop your pipeline
  if (m_pipelineLoopTask != nullptr)
    m_pipelineLoopTask->stop();
  return FrameworkReturnCode::_SUCCESS;
}

SinkReturnCode MyPipeline::update(Transform3Df& pose)
{
  // ADD HERE the code to update the pose of the camera

  return SinkReturnCode::_NOT_IMPLEMENTED;
}

}
}
1 Add the header file for the data exchange between your component. Here, an image exhnage between the camera and the image viewer.
2 We declare the injection of the 3 components required for this pipeline.
3 A simple code to get camera parameters and return it.
4 Implementation of the pipeline loop. If an input image has been updated, we will access to it through the source component and we pass it to the sink component. If not, we read the last image captured by the camera and we pass it to the sink component.
5 Set the m_externalInputImageMode to true to inform the pipeline that it run with external input images, and put the image in the buffer of the source component.
6 Set the image buffer for the sink component.
7 Start the camera.

Add your pipeline to your module

To add your pipeline to your module, you have just to edit the file myPipeline_main.cpp:

myPipeline_main.cpp
#include <xpcf/module/ModuleFactory.h>
#include "MyPipeline.h" (1)
#include <iostream>

namespace xpcf=org::bcom::xpcf;

/**
 *  @ingroup xpcfmodule
 */
/**
  * Declare module.
  */
XPCF_DECLARE_MODULE("{df891348-4683-432d-beff-fb9ead08f020}","SolAR::PIPELINES::MYPIPELINE","MyPipeline module description");

/**
 * This method is the module entry point.
 * XPCF uses this method to create components available in the module.
 *
 * Each component exposed must be declared inside a xpcf::tryCreateComponent<ComponentType>() call.
 */
extern "C" XPCF_MODULEHOOKS_API xpcf::XPCFErrorCode XPCF_getComponent(const xpcf::uuids::uuid& componentUUID,SRef<xpcf::IComponentIntrospect>& interfaceRef)
{
    xpcf::XPCFErrorCode errCode = xpcf::XPCFErrorCode::_FAIL;
    // Sample code to declare components instanciation
    errCode = xpcf::tryCreateComponent<SolAR::PIPELINES::MyPipeline>(componentUUID,interfaceRef); (2)
    /*
    if (errCode != xpcf::XPCFErrorCode::_SUCCESS) {
        errCode = xpcf::tryCreateComponent<SolAR::PIPELINES::otherComponentType>(componentUUID,interfaceRef);
    }
    */
    return errCode;
}

/**
  * The declarations below populate list of the components available in the module (it represents the module index).
  * XPCF uses this index to introspect the components available in a module, providing the ability to generate the configuration file skeleton from the code.
  */
XPCF_BEGIN_COMPONENTS_DECLARATION
// sample components declarations
XPCF_ADD_COMPONENT(SolAR::PIPELINES::MyPipeline) (3)
//XPCF_ADD_COMPONENT(SolAR::PIPELINES::otherComponentType)
XPCF_END_COMPONENTS_DECLARATION
1 Add the header file of your pipeline.
2 Add this line to be able to let XPCF create your pipeline component.
3 Add you pipeline to the XPCF components declaration.

Now you can rebuild your module (do not forget if not on Windows with QTVS mode to add a make install for debug and release mode in the configuration of your QT project). Your pipeline is now ready to use, and should be available in your remaken packages folder.

Export your pipeline for Unity

General

First of all, you must follow the installation instructions of Unity and of the SolAR plugin for unity available in the section Use Unity.

Then, add your pipeline in the packagedependencies.txt available in the root folder of the SolARUnityProject:

SolARPipeline_FiducialMarker|0.11.0|SolARPipeline_FiducialMarker|SolARBuild@github|https://github.com/SolarFramework/Sample-FiducialMarker/releases/download
SolARPipeline_NaturalImageMarker|0.11.0|SolARPipeline_NaturalImageMarker|SolARBuild@github|https://github.com/SolarFramework/Sample-NaturalImageMarker/releases/download
SolARPipeline_SLAM|0.11.0|SolARPipeline_SLAM|SolARBuild@github|https://github.com/SolarFramework/Sample-Slam/releases/download
SolARPipelineManager|0.11.0|SolARPipelineManager|SolARBuild@github|https://github.com/SolarFramework/SolARPipelineManager/releases/download
SolARModuleOpenCV|0.11.0|SolARModuleOpenCV|SolARBuild@github|https://github.com/SolarFramework/SolARModuleOpenCV/releases/download
SolARModuleTools|0.11.0|SolARModuleTools|SolARBuild@github|https://github.com/SolarFramework/SolARModuleTools/releases/download
SolARWrapper|0.11.0|SolARWrapper|SolARBuild@github|https://github.com/SolarFramework/SwigWrapper/releases/download
SolARModuleFBOW|0.11.0|SolARModuleFBOW|SolARBuild@github|https://github.com/SolarFramework/SolARModuleFBOW/releases/download
SolARModuleG2O|0.11.0|SolARModuleG2O|SolARBuild@github|https://github.com/SolarFramework/SolARModuleG2O/releases/download
MyPipeline|0.11.0|MyPipeline|@github|https://github.com/SolarFramework/MyPipeline/releases/download (1)
1 The line to add your pipeline to the packagedependencies.txt available in teh root folder of the SolARUnityProject. Ideally, the binaries of your pipeline should be uploaded in the release of GitHub or on an artifactory.

Then, run the install.bat available in the root folder of the SolARUnityProject (available only on Windows). This script should install all required binaries in the Assets/Plugins folder of the unity project.

Copy also your configuration file MyPipelineConfiguration.xml in the Assets/SolAR/Pipelines folder of your unity project, and replace the opencv module path $XPCF_MODULE_ROOT/SolARBuild/SolARModuleOpenCV/0.10.0/lib/x86_64/shared by /Assets/Plugins as now you will use the module libraris copy locally in your Unity project (do it for all modules in your configuration file).

Now, in the Unity editor, you can select again your pipeline folder in the inspector of your SolARPipelineLoader, and select in the pipeline list your pipeline.

If you want to export your pipeline, create a new Unity package with:

  • The shared binaries stored in the Unity plugin folder relative to:

    • Your pipeline

    • The modules used by your pipeline

    • The third parties of the modules used by your pipeline

  • The pipleline configuration file stored under Assets/SolAR/Pipelines/.

Unity Android Deployment

For Unity platform we proceed to a change in the path of the pipeline configuration file to match applications public path. Your pipeline configuration will be available from the device and could be edited directly to manage its components, parameters, etc.

Deployment details

/!\ Only arm64-v8a architecture are supported by SolAR.

While you launch build for Android platform :

  • On Unity

    • Unity pre-build process will be called to generate ./Assets/StreamingAssets/SolAR/Android/android.xml. This file will list every file in under your ./Assets/StreamingAssets/.

    • Path of pipelines xml in ./Assets/StreamingAssets/SolAR/Pipelines/ are set to match Android filesystem to the public application directory. This will let your xml available on your device and you could edit them.

    • This file will be read to clone every asset included in the Android private JAR to Android application public path

    • APK is built and it includes Unity StreamingAssets and libraries

  • On Android

    • While the application is launched for the first time, it will look for android.xml in the private application JAR. Then if this file doesn’t exist in the device application public path (/storage/emulated/0/Android/com.bcom/SolARDemo/files/StreamingAssets/SolAR/Android) it will be read and all of his content lists will be cloned into this path. Otherwise, the already present one will be read and cloned. If you don’t want to overwrite an asset at each application launching you can set the overwrite attribute for a dedicated file to false.

    • The pipeline selected by the application will be load (from public application path) and his path to ./Assets/Plugins will be matched with Android private JAR path (/data/app/com.bcom/SolARUnityPlugin-[only-known-on-running]==/lib/arm64/).

    • Application is initialized correctly. You can change the pipeline selected with SolARMenu in the right-hand corner.

Android pipeline
Figure 1. Android pipeline

Test your pipeline in C++

To test your pipeline in C++, have a look to the section use in C++.