What is a C++ pipeline for Unity ?

A C pipeline is a shared library embedding an implementation of the IPipeline interface defined by the SolAR framework. Thus, this pipeline can be loaded at run-time and run by any third party application such as a C application or by Unity. To provide configuration capabilities to a pipeline, it is considered by SolAR as a component embedded in a module. for this reason, readers are encouraged to have a look at the create a module section.

While SolAR is dedicated to computer vision pipelines, for now, the pipeline interface is only dedicated to pose estimation pipeline. To be more specific, the pipelines built to be used in Unity are pipelines that take on hand camera frames and that deliver on the other hand camera pose estimation.

New pipeline interfaces will be soon added for other viion task such as 3D reconstruction, object recognition, etc.

Initialize your pipeline step-by-step_

Create your pipeline project

QT Creator

Open QTCreator and create a new project (in File menu).

Select Library and C++ Library and click on Choose button.

create a shared C++ library in QT
Figure 1. Project creation

Then, set the name of your project with the name of your pipeline, set the location (a dedicated folder with all pipeline projects is recommended) and select "Shared Library" for the type of the library.

select library type
Figure 2. Project name, location and library type

Next, select your development kits. We recommend to use MSVC 2017 64bit or Clang. Then, unselect all required modules. On the next window, keep the name of the class to create with the name of your module. Unfortunately, with the QT project creation wizard, you cannot set dedicated folders for headers and sources. As we recommend to put the header files in an "interfaces" folder and the source files in a "src" folder, you will have to create these two folders manually and move the two files (MyPipeline.h and MyPipeline.cpp) in their respective folder (interfaces and src). Finally go to the summary and click on "Finish".

If QT Creator has created a file called mypipeline_global.h, supress it.

SolAR provide a smart and easy to use build and deployment system.

Now, you need to select the file called "MyPipeline.pro" in your project tree, and replace the script by the following one:

MyPipeline.pro
## remove Qt dependencies
QT       -= core gui
CONFIG -= qt

## global defintions : target lib name, version
TARGET = MyPipeline (1)
FRAMEWORK = $$TARGET
VERSION=x.x.x (2)

DEFINES += MYVERSION=$${VERSION}
DEFINES += TEMPLATE_LIBRARY
CONFIG += c++1z


CONFIG(debug,debug|release) {
    DEFINES += _DEBUG=1
    DEFINES += DEBUG=1
}

CONFIG(release,debug|release) {
    DEFINES += _NDEBUG=1
    DEFINES += NDEBUG=1
}

DEPENDENCIESCONFIG = shared recurse

include (../../builddefs/qmake/templatelibconfig.pri) (5)

## DEFINES FOR MSVC/INTEL C++ compilers
msvc {
DEFINES += "_BCOM_SHARED=__declspec(dllexport)"
}

INCLUDEPATH += interfaces/

HEADERS += interfaces/MyPipeline.h (3)

SOURCES += src/MyPipeline.cpp (4)

unix {
    QMAKE_CXXFLAGS += -Wignored-qualifiers
    QMAKE_LINK=clang++
    QMAKE_CXX = clang++
}

macx {
    DEFINES += _MACOS_TARGET_
    QMAKE_MAC_SDK= macosx
    QMAKE_CFLAGS += -mmacosx-version-min=10.7 -std=c11 #-x objective-c++
    QMAKE_CXXFLAGS += -mmacosx-version-min=10.7 -std=c11 -std=c++11 -O3 -fPIC#-x objective-c++
    QMAKE_LFLAGS += -mmacosx-version-min=10.7 -v -lstdc++
    LIBS += -lstdc++ -lc -lpthread
}

win32 {

    DEFINES += WIN64 UNICODE _UNICODE
    QMAKE_COMPILER_DEFINES += _WIN64
    QMAKE_CXXFLAGS += -wd4250 -wd4251 -wd4244 -wd4275
}

header_files.path = $${PROJECTDEPLOYDIR}/interfaces
header_files.files = $$files($${PWD}/interfaces/*.h*)

# HOME variable =  C:\User\<yourUserName>
xpcf_xml_files.path = $$(HOME)/.xpcf/SolAR
xpcf_xml_files.files=$$files($${PWD}/xpcf*.xml)

INSTALLS += header_files
INSTALLS += xpcf_xml_files

Now, just update the .pro file:

1 replace the TARGET by the name of your pipeline,
2 set the version number of your pipeline,
3 change the name of your header file when it is added to the HEADERS variable,
4 change the name of your source file when it is added to the SOURCES_ variable.
5 check if the builddefs folder used to define the compilation pipeline is well referenced
In order to deploy your pipeline in the binary folder ${REMAKENROOT]/MyPipeline/MyPipelineVersion after building, in QT creator click in the left menu on Projects, then on Build, on add a build step and select Make. Set the Make Argument to the value install. Do it for both release and debug mode.

 — Using Cmake instead of .pro QT file — 

Create a CMakeLists.txt file and copy the following code to it. This file use temporarily BCOMDEVROOT environment variable link to your SoLAR sources.

CMakeLists.txt
cmake_minimum_required(VERSION 3.7.2)

##################################################
set (VERSION_NUMBER "x.x.x") (2)
project("MyPipeline") (1)

set (HEADERS
    interfaces/MyPipeline.h
) (3)

set (SOURCES
    src/MyPipeline.cpp
) (4)

install (FILES "${CMAKE_CURRENT_SOURCE_DIR}/MyPipelineConfiguration.xml" DESTINATION $ENV{BCOMDEVROOT}/bcomBuild/$ENV{CMAKE_PROJECT_NAME}/$ENV{PROJECT_VERSION}/	)  (5)

##################################################

# various macros
include("$ENV{BCOMDEVROOT}/bcomBuild/SolARFramework/solarmacros.cmake") (6)
# config setup
setup()
# process packagedependencies.txt
processPackagedependencies()
# define targets (type library)
defineTargets("library" "")

now, just update the CMakeLists.txt file:

1 set the name of the project with the name of your module,
2 set the version number of your module,
3 change the name of your header file when it is added to the HEADERS variable,
4 change the name of your source file when it is added to the SOURCES_ variable.
5 update with the name of your pipeline configuration file to copy it from your pipeline project to your deployment directory ${BCOMDEVROOT}/MyPipeline/MyPipelineVersion/.
6 check that the solarmacros.cmake file exists. This file must be install when you have launched the installer. If you decide to build the framework by yourself, this file will be copied in your ${BCOMDEVROOT} folder when you will build and install the SolARFramework project.

Select your dependencies

As mentionned previously, SolAR framework provides developers with a building pipeline allowing among other things to easily manage dependencies (download, version management, packaging during deployment step, etc.).

To define the dependencies used by your pipeline, create a file called packagedependencies.txt at the root of your project.

Following, an example of packagedependencies.txt file for a standalone pipeline using OpenCV module:

/.packagedependencies.txt

xpcf|2.2.0|xpcf|artifactory|https://repository.b-com.com/amc-generic
spdlog|0.14.0|spdlog|thirdParties|https://github.com/SolarFramework/binaries/releases/download
eigen|3.3.5|eigen|thirdParties|https://github.com/SolarFramework/binaries/releases/download
SolARFramework|0.6.0|SolARFramework|github|https://github.com/SolarFramework/SolarFramework/releases/download
SolARModuleTools|0.6.0|SolARModuleTools|github|https://github.com/SolarFramework/SolARModuleTools/releases/download
SolARModuleOpenCV|0.6.0|SolARModuleOpenCV|github|https://github.com/SolarFramework/SolARModuleOpenCV/releases/download
opencv|3.4.3|opencv|thirdParties|https://github.com/SolarFramework/binaries/releases/download

Here is the syntax for each dependency:

Directory|version|name|local location|remote location

artifactory refers to .remaken/packages/<yourCompiler>/ folder. You can create a REMAKENROOT variable. All process are made in this folder.

If you need new modules please refer to the Get modules section, and if you need more third parties, please refer to the Package your third parties section.

xpcf, boost, spdlog, eigen and SolARFramework are mandatory third parties for assembling a pipeline.

Numerous samples of packagedependencies.txt files can be found with the SolAR Samples you have surely installed on your machine.

Create the configuration file

his file will be used at run-time. It will allow XPCF to load Module components and configure them at run-time. So you can experiment different implementations and configurations of a pipeline without the need to recompile your application.

Creating a configuration file is very easy. Once you have identified the modules and the components required to assemble your pipeline, just record them in the xml configuration file.

Following, an example of a configuration file for a pipeline using the camera and image viewer components embedded in the SolARModuleOpenCV module :

MyPipelineConfiguration.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<xpcf-registry autoAlias="true">

<module uuid="94b21be9-703b-4a00-86a9-0db1bf70ef89" name="MyPipelineModule" path="$REMAKENROOT/MyPipeline/0.0.1/lib/x86_64/shared" description="MyPipelineModule">
  <component uuid="855c83b7-f4ec-48ab-8e89-56018ea9e169" name="MyPipeline" description="MyPipeline">
    <interface uuid="125f2007-1bf9-421d-9367-fbdc1210d006" name="IComponentIntrospect" description="IComponentIntrospect"/>
    <interface uuid="b5a6225e-6a91-4050-b298-886f4c17d9d2" name="IPipeline" description="IPipeline"/>
  </component>
</module>
<module uuid="15e1990b-86b2-445c-8194-0cbe80ede970" name="SolARModuleOpenCV" path="$REMAKENROOT/SolARModuleOpenCV/0.6.0/lib/x86_64/shared" description="OpenCV">
  <component uuid="5B7396F4-A804-4F3C-A0EB-FB1D56042BB4" name="SolARCameraOpencv" description="SolARCameraOpencv">
    <interface uuid="125f2007-1bf9-421d-9367-fbdc1210d006" name="IComponentIntrospect" description="IComponentIntrospect"/>
    <interface uuid="5DDC7DF0-8377-437F-9C81-3643F7676A5B" name="ICamera" description="ICamera"/>
  </component>
</module>
<module uuid="28b89d39-41bd-451d-b19e-d25a3d7c5797" name="SolARModuleTools" path="$REMAKENROOT/SolARModuleTools/0.6.0/lib/x86_64/shared" description="Module Tools">
  <component uuid="85db2f25-4f1c-4e06-9011-e020284bfc4f" name="SolARBasicSink" description="A Sink component for a synchronized pose and image pair">
    <interface uuid="125f2007-1bf9-421d-9367-fbdc1210d006" name="IComponentIntrospect" description="IComponentIntrospect" />
    <interface uuid="c0d9fee4-d7d7-4866-a6cd-3bacac23316a" name="ISinkPoseImage" description="An interface allowing to store a synchronized pose and image from the pipeline to make it available to a third party application" />
  </component>
  <component uuid="1e43cda9-7850-4a8a-a32b-f3f31ea94902" name="SolARBasicSource" description="A source component for feeding the pipeline with external images">
    <interface uuid="125f2007-1bf9-421d-9367-fbdc1210d006" name="IComponentIntrospect" description="IComponentIntrospect" />
    <interface uuid="06e2fc5d-39da-4486-b2a6-1d8bd788fa13" name="ISourceImage" description="An interface allowing to store an image from extern source to make it available to the pipeline" />
  </component>
</module>

<properties>
  <configuration component="SolARCameraOpencv>
    <property name="calibrationFile" type="string" value="camera_calibration.yml"/>
    <property name="deviceID" type="UnsignedInteger" value="0"/>
  </configuration>
</properties>

</xpcf-registry>
1 Add the modules used by your pipeline. To get information concerning the modules, have a look to their registry file available in the ${REMAKENROOT}/"ModuleName"/"ModuleVersion"/.
2 For each module, add the components used by your pipeline. To get information concerning the components, have a look to the registry file of their module available in the ${REMAKENROOT}/"ModuleName"/"ModuleVersion"/.
3 Optionally, add a configuration to your components. The name of your configuration parameters are generally the name of the variable in the class definition of the component without the prefix "m_". The type of the configuration parameters are simple types such as "String", "Integer", "Float", "Double", or array of these simple types.

Examples of such configuration files are available in the samples directory that is provided with the SolAR installation.

Implement your pipeline

IPipeline API

A virtual class named IPipeline has been defined in SolAR framework and is located in api/pipeline/IPipeline.h .
Any SolAR pipeline should implement this class, that means that six methods are to be implemented in the corresponding source code :

  • init: Initialize the pipeline by providing a reference to the component manager loaded by the PipelineManager. You will have to instantiate all the components used by your pipeline and initialize by for instance starting the camera, initialize component with intrinsic parameter of the camera, etc.

  • getCameraParameters: Provide third party applications with the calibration parameters of the camera. Useful hen you want to set the parameter of the virtual camera with the ones from the real one.

  • start: Start the pipeline with a texture buffer that will be updated when a new frame will be processed and ready for display.

  • stop: Stop the pipeline.

  • update: A methode that provide the new pose of the camera.

  • loadSourceImage: If there is no camera in your pipeline, you can feed it with an external image (for instance, an image capture by the Unity web camera).

A full description of these methods is available in the api/pipeline/IPipeline.h file.

Pipeline Template

To implement your pipeline, you have to fill in the header file MyPipeline.h and the source file MyPipeline.cpp. Next, we provide you with templates for these to files.

MyPipeline.h
/**
 * Information concerning the copyright and licence of your pipeline
 */

#ifndef MYPIPELINE_H
#define MYPIPELINE_H

(1)
#if _WIN32
#ifdef MyPipeline_API_DLLEXPORT
#define MYPIPELINE_EXPORT_API __declspec(dllexport)
#else //MYPIPELINE_API_DLLEXPORT
#define MYPIPELINE_EXPORT_API __declspec(dllimport)
#endif //MYPIPELINE_API_DLLEXPORT
#else //_WIN32
#define MYPIPELINE_EXPORT_API
#endif //_WIN32

// Mandatory include file (2)
#include "xpcf/core/traits.h"
#include "xpcf/component/ConfigurableBase.h"
#include "api/pipeline/IPipeline.h"
#include "xpcf/threading/BaseTask.h"

// Add here the header file for the data structures required by your pipeline
// e.g. #include "datastructure/Image.h"
(3)

// Add here the header file for the component interfaces required by your pipeline
// e.g. #include "api/input/devices/ICamera.h"
(4)

namespace SolAR { (5)
using namespace datastructure;
using namespace api;
namespace PIPELINES {

/**
 * @class MyPipeline
 * @brief A short description of your pipeline
 */
(6)
class MYPIPELINE_EXPORT_API MyPipeline : public org::bcom::xpcf::ConfigurableBase,
    public api::pipeline::IPipeline
{
public:
    MyPipeline();
    ~MyPipeline();

    //// @brief Initialization of the pipeline
    /// Initialize the pipeline by providing a reference to the component manager loaded by the PipelineManager.
    /// @param[in] componentManager a shared reference to the component manager which has loaded the components and configuration in the pipleine manager
    FrameworkReturnCode init(SRef<xpcf::IComponentManager> xpcfComponentManager) override;

    /// @brief Provide the camera parameters
    /// @return the camera parameters (its resolution and its focal)
    pipeline::CameraParameters getCameraParameters() override;

    /// @brief Starts the pipeline and provides a texture buffer which will be updated when required.
    /// @param[in] textureHandle a pointer to the texture buffer which will be updated at each call of the update method.
    FrameworkReturnCode start(void* imageDataBuffer) override;

    /// @brief Stop the pipeline.
    FrameworkReturnCode stop() override;

    /// @brief update the pipeline
    /// Get the new pose and update the texture buffer with the image that has to be displayed
    SinkReturnCode update(Transform3Df& pose) override;

    /// @brief load the source image
    SourceReturnCode loadSourceImage(void* sourceTextureHandle, int width, int height) override;

    void unloadComponent () override final;

private:
  // Add here the declaration of the shared references on components required by your pipeline (e.g. SRef<input::device::ICamera> m_camera;)
  (7)

  // Threads (8)
  bool pipelineLoop();
  xpcf::DelegateTask* m_pipelineLoopTask = nullptr;

  // optionally, add here some parameters of you rmodule that can be configured
  // int m_myPipelineParameter; (9)
};

}
}


XPCF_DEFINE_COMPONENT_TRAITS(SolAR::PIPELINES::MyPipeline, (10)
                             "<yourUUID>",
                             "My pipeline",
                             "A sample pipeline used for documentation");

#endif // MYPIPELINE_H
1 Add the required declarations to export the pipeline functions embedded in your shared library. Just replace in a case sensitive manner "MyModule" by the name of your module.
2 These include files are mandatory. They correspond to the xpcf traits file as here we declare the module traits directly in this file, to the XPCF configurableBase header file if you want to configure your pipeline through an external xml file, and to the SolAR IPipeline header file, as your pipeline implement this SolAR interface.
3 The header files of the SolAR data structures required to connect your components.
4 The header files of the SolAR component interfaces required by your pipeline.
5 SolAR and XPCF namespaces directives to shorten the calls to SolAR api and data structures.
6 Declare your pipeline class, and add the functions defined as abstract in the IPipeline interface.
7 Declaration of the components used by the pipeline (thanks to their abstract interfaces). We are using a shared reference of the components to ease their use in multi-threaded pipelines.
8 [white]#Declare a xpcf task and the function running the pipeline loop in a dedicated thread.
9 You can declare parameters for the pipeline. Some of these parameters may be configurable.
10 Finally, add the trait of your pipeline. To generate an UUID, you can use a online UUID generator
MyPipeline.cpp
/**
 * Information concerning the copyright and licence of your pipeline
 */

// Mandatory header file (1)
#include "xpcf/module/ModuleFactory.h"
#include "core/Log.h"

// Header to your pipeline declaration (2)
#include "MyPipeline.h"

// Add here the headers to the traits of the module used by your pipeline (3)
// e.g. #include "SolARModuleOpencv_traits.h"

namespace xpcf=org::bcom::xpcf;

// Declaration of the module embedding MyPipeline (4)
XPCF_DECLARE_MODULE("<yourUUID>", "MyPipelineModule", "The module embedding MyPipeline")

// Add this function required by the XPCF component factory (5)
extern "C" XPCF_MODULEHOOKS_API xpcf::XPCFErrorCode XPCF_getComponent(const boost::uuids::uuid& componentUUID,SRef<xpcf::IComponentIntrospect>& interfaceRef)
{
    xpcf::XPCFErrorCode errCode = xpcf::XPCFErrorCode::_FAIL;
    errCode = xpcf::tryCreateComponent<SolAR::PIPELINES::MyPipeline>(componentUUID,interfaceRef);

    return errCode;
}

// Add the declaration of the component (here a pipeline) to XPCF (6)
XPCF_BEGIN_COMPONENTS_DECLARATION
XPCF_ADD_COMPONENT(SolAR::PIPELINES::MyPipeline)
XPCF_END_COMPONENTS_DECLARATION

// Macro used by the XPCF factory to define a component/pipeline (7)
XPCF_DEFINE_FACTORY_CREATE_INSTANCE(SolAR::PIPELINES::MyPipeline)

namespace SolAR { (8)
using namespace datastructure;
using namespace api::pipeline;
namespace PIPELINES {

MyPipeline::MyPipeline():ConfigurableBase(xpcf::toUUID<MyPipeline>())
{
   addInterface<api::pipeline::IPipeline>(this);
   // Add here the mapping to make configurable some attributes of your pipeline
   // SRef<xpcf::IPropertyMap> params = getPropertyRootNode();
   // params->wrapInteger("myPipelineParameter", m_myPipelineParameter);
   (9)
}

MyPipeline::~MyPipeline()
{

}

bool MyPipeline::pipelineLoop() (10)
{
  // ADD HERE the code connecting the component of your pipeline

  return true;
}

FrameworkReturnCode MyPipeline::init(SRef<xpcf::IComponentManager> xpcfComponentManager)
{
  try {
    // ADD HERE: instantiate concrete components and bind them to abstract component interfaces
    (11)
    // e.g. SRef<image::ICamera> camera = xpcfComponentManager->create<SolAR::MODULES::OPENCV::SolARCameraOpencv>()->bindTo<image::ICamera>();

  }
  catch (xpcf::Exception e)
  {
     LOG_WARNING("One or more components cannot be created: {}", e.what());
     return FrameworkReturnCode::_ERROR_;
  }

  // ADD HERE: The pipeline initialization
  (12)
  return FrameworkReturnCode::_SUCCESS;
}

CameraParameters MyPipeline::getCameraParameters()
{
  CameraParameters camParam;
  // ADD HERE the code to return camera parameters

  return camParam;
}

SourceReturnCode MyPipeline::loadSourceImage(void* sourceTextureHandle, int width, int height)
{
   // ADD HERE the code to take external image as input of your pipeline

   return SourceReturnCode::_NOT_IMPLEMENTED;
}


FrameworkReturnCode MyPipeline::start(void* imageDataBuffer)
{
  // ADD HERE the code to start your pipeline

  // create and start a thread for the loop of the pipeline (13)
  auto pipelineLoopThread = [this](){;pipelineLoop();};
  m_pipelineLoopTask = new xpcf::DelegateTask(pipelineLoopThread);
  m_pipelineLoopTask->start();

  return FrameworkReturnCode::_SUCCESS;
}

FrameworkReturnCode MyPipeline::stop()
{
  // ADD HERE the code to stop your pipeline
  if (m_pipelineLoopTask != nullptr)
    m_pipelineLoopTask->stop(); (14)
  return FrameworkReturnCode::_SUCCESS;
}

SinkReturnCode MyPipeline::update(Transform3Df& pose)
{
  // ADD HERE the code to update the pose of the camera

  return SinkReturnCode::_NOT_IMPLEMENTED;
}

}
}
1 Required header file corresponding to the XPCF module factory.
2 Obviously, add the header file corresponding to your pipeline declaration.
3 Add the trait header files for the modules required by your pipeline.
4 Declare your module to XPCF. generate a UUID for your module by using an online UUID generator.
5 Implement the method XPCF_getComponent required by the XPCF factory to instantiate the componen t(here a pipeline).
6 Add your component/pipeline to XPCF.
7 Add the definition of your pipeline to the XPCF component factory.
8 Embed your pipeline in the SolAR/PIPELINES namespace.
9 To add a configurable parameter and map it to the xml configuration file, retrieve a reference to the properties of the component/pipeline, and add a wrapper for each configuration parameter you want to map. Here, when an xml element with the name "myPipelineParameter" of type integer will be specified in the configuration file, when exiting the constructor, its value will be set to the attribute m_myPipelineParameter.
10 Implement the loop of the pipeline by connecting components. This loop will run in a dedicated thread.
11 Instantiate all components required by your pipeline here.
12 If required, add the code to initialize your pipeline (e.g. load a reference image, initialize some component with the intrinsic parameters of the camera, etc.).
13 Start the pipeline loop thread when the start method is called.
14 Stop the pipeline loop thread when the stop method is called.

Pipeline sample

We will present next the implementation of the simplest pipeline that consists in capturing an image from a camera and make it available for a third party application. For this implementation, we will need only one module: SolARModuleOpenCV.

MyPipeline.h
/**
 * Information concerning the copyright and licence of your pipeline
 */

#ifndef MYPIPELINE_H
#define MYPIPELINE_H

#if _WIN32
#ifdef MyPipeline_API_DLLEXPORT
#define MYPIPELINE_EXPORT_API __declspec(dllexport)
#else //MYPIPELINE_API_DLLEXPORT
#define MYPIPELINE_EXPORT_API __declspec(dllimport)
#endif //MYPIPELINE_API_DLLEXPORT
#else //_WIN32
#define MYPIPELINE_EXPORT_API
#endif //_WIN32

// Mandatory include file
#include "xpcf/core/traits.h"
#include "xpcf/component/ConfigurableBase.h"
#include "api/pipeline/IPipeline.h"
#include "xpcf/threading/BaseTask.h"

// Add here the header file for the data structures required by your pipeline
// e.g. #include "datastructure/Image.h"
#include "datastructure/Image.h" (1)

// Add here the header file for the component interfaces required by your pipeline
// e.g. #include "api/input/devices/ICamera.h" (2)
#include "api/input/devices/ICamera.h"
#include "api/sink/ISinkPoseImage.h"
#include "api/source/ISourceImage.h"



namespace SolAR {
using namespace datastructure;
using namespace api;
namespace PIPELINES {

/**
 * @class MyPipeline
 * @brief A short description of your pipeline
 */
class MYPIPELINE_EXPORT_API MyPipeline : public org::bcom::xpcf::ConfigurableBase,
    public api::pipeline::IPipeline
{
public:
    MyPipeline();
    ~MyPipeline();

    //// @brief Initialization of the pipeline
    /// Initialize the pipeline by providing a reference to the component manager loaded by the PipelineManager.
    /// @param[in] componentManager a shared reference to the component manager which has loaded the components and configuration in the pipleine manager
    FrameworkReturnCode init(SRef<xpcf::IComponentManager> xpcfComponentManager) override;

    /// @brief Provide the camera parameters
    /// @return the camera parameters (its resolution and its focal)
    pipeline::CameraParameters getCameraParameters() override;

    /// @brief Starts the pipeline and provides a texture buffer which will be updated when required.
    /// @param[in] textureHandle a pointer to the texture buffer which will be updated at each call of the update method.
    FrameworkReturnCode start(void* imageDataBuffer) override;

    /// @brief Stop the pipeline.
    FrameworkReturnCode stop() override;

    /// @brief update the pipeline
    /// Get the new pose and update the texture buffer with the image that has to be displayed
    SinkReturnCode update(Transform3Df& pose) override;

    /// @brief load the source image
    SourceReturnCode loadSourceImage(void* sourceTextureHandle, int width, int height) override;

    void unloadComponent () override final;

private:
  // Add here the declaration of the shared references on components required by your pipeline (e.g. SRef<input::device::ICamera> m_camera;)
  SRef<input::devices::ICamera> m_camera; (3)
  SRef<sink::ISinkPoseImage> m_sink;
  SRef<source::ISourceImage> m_source;

  // Threads
  bool pipelineLoop();
  xpcf::DelegateTask* m_pipelineLoopTask = nullptr;

  // optionally, add here some parameters of your module that can be configured
  // int m_myPipelineParameter;

  // Other attributes
  bool m_externalInputImageMode = false; (4)
};

}
}


XPCF_DEFINE_COMPONENT_TRAITS(SolAR::PIPELINES::MyPipeline,
                             "855c83b7-f4ec-48ab-8e89-56018ea9e169",
                             "My pipeline",
                             "A sample pipeline used for documentation");

#endif // MYPIPELINE_H
1 include the header file of a SolAR image to connect the camera component to the sink component.
2 include the component interface headers for a camera component, a sink component and a source component. A sink component handles an output buffer feed by the pipeline a read by external third parties. Reciprocally, a source component handles an input buffer feeds by external third parties and used by the pipeline.
3 Declaration of the three components used by this pipeline, a camera, a sink component handling the pose and the output image, and a source component handling an input image.
4 Add an attribute to know if the pipeline use as input an image coming from a third party.
MyPipeline.cpp
/**
 * Information concerning the copyright and licence of your pipeline
 */

// Mandatory header file
#include "xpcf/module/ModuleFactory.h"
#include "core/Log.h"

// Header to your pipeline declaration
#include "MyPipeline.h"

// Add here the headers to the traits of the module used by your pipeline
// e.g. #include "SolARModuleOpencv_traits.h"
#include "SolARModuleOpencv_traits.h" (1)
#include "SolARModuleTools_traits.h"

namespace xpcf=org::bcom::xpcf;

// Declaration of the module embedding MyPipeline
XPCF_DECLARE_MODULE("855c83b7-f4ec-48ab-8e89-56018ea9e169", "MyPipelineModule", "The module embedding MyPipeline")

// Add this function required by the XPCF component factory
extern "C" XPCF_MODULEHOOKS_API xpcf::XPCFErrorCode XPCF_getComponent(const boost::uuids::uuid& componentUUID,SRef<xpcf::IComponentIntrospect>& interfaceRef)
{
    xpcf::XPCFErrorCode errCode = xpcf::XPCFErrorCode::_FAIL;
    errCode = xpcf::tryCreateComponent<SolAR::PIPELINES::MyPipeline>(componentUUID,interfaceRef);

    return errCode;
}

// Add the declaration of the component (here a pipeline) to XPCF
XPCF_BEGIN_COMPONENTS_DECLARATION
XPCF_ADD_COMPONENT(SolAR::PIPELINES::MyPipeline)
XPCF_END_COMPONENTS_DECLARATION

// Macro used by the XPCF factory to define a component/pipeline
XPCF_DEFINE_FACTORY_CREATE_INSTANCE(SolAR::PIPELINES::MyPipeline)

namespace SolAR {
using namespace datastructure;
using namespace api::pipeline;
namespace PIPELINES {

MyPipeline::MyPipeline():ConfigurableBase(xpcf::toUUID<MyPipeline>())
{
   addInterface<api::pipeline::IPipeline>(this);
   // Add here the mapping to make configurable some attributes of your pipeline
   // SRef<xpcf::IPropertyMap> params = getPropertyRootNode();
   // params->wrapInteger("myPipelineParameter", m_myPipelineParameter);
}

MyPipeline::~MyPipeline()
{

}

bool MyPipeline::pipelineLoop() (2)
{
  // ADD HERE the code connecting the component of your pipeline
  SRef<Image> image;

  if (m_externalInputImageMode)
  {
    if (m_source->getNextImage(image) == SourceReturnCode::_NEW_IMAGE)
      m_sink->set(image);
  }
  else
  {
    if (m_camera->getNextImage(image) == SolAR::FrameworkReturnCode::_ERROR_LOAD_IMAGE)
      return false;
    m_sink->set(image);
  }
  return true;
}

FrameworkReturnCode MyPipeline::init(SRef<xpcf::IComponentManager> xpcfComponentManager)
{
  try {
    // ADD HERE: instantiate concrete components and bind them to abstract component interfaces
    // e.g. SRef<image::ICamera> camera = xpcfComponentManager->create<SolAR::MODULES::OPENCV::SolARCameraOpencv>()->bindTo<image::ICamera>();
    (3)
    m_camera = xpcfComponentManager->create<MODULES::OPENCV::SolARCameraOpencv>()->bindTo<input::devices::ICamera>();
    m_sink = xpcfComponentManager->create<MODULES::TOOLS::SolARBasicSink>()->bindTo<sink::ISinkPoseImage>();
    m_source = xpcfComponentManager->create<MODULES::TOOLS::SolARBasicSource>()->bindTo<source::ISourceImage>();
  }
  catch (xpcf::Exception e)
  {
     LOG_WARNING("One or more components cannot be created: {}", e.what());
     return FrameworkReturnCode::_ERROR_;
  }

  // ADD HERE: The pipeline initialization

  return FrameworkReturnCode::_SUCCESS;
}

CameraParameters MyPipeline::getCameraParameters()
{
  CameraParameters camParam;
  // ADD HERE the code to return camera parameters
  if (m_camera) (4)
    {
        camParam  m_camera->getCameraParameters();
    }
  return camParam;
}

SourceReturnCode MyPipeline::loadSourceImage(void* sourceTextureHandle, int width, int height)
{
   // ADD HERE the code to take external image as input of your pipeline
   m_externalInputImageMode = true; (5)
   return m_source->setInputTexture((unsigned char *)sourceTextureHandle, width, height);
}


FrameworkReturnCode MyPipeline::start(void* imageDataBuffer)
{
  // ADD HERE the code to start your pipeline
  m_sink->setImageBuffer((unsigned char*)imageDataBuffer); (6)

  if (m_camera->start() != FrameworkReturnCode::_SUCCESS) (7)
    return FrameworkReturnCode::_ERROR_;

  // create and start a thread for the loop of the pipeline
  auto pipelineLoopThread = [this](){;pipelineLoop();};
  m_pipelineLoopTask = new xpcf::DelegateTask(pipelineLoopThread);
  m_pipelineLoopTask->start();

  return FrameworkReturnCode::_SUCCESS;
}

FrameworkReturnCode MyPipeline::stop()
{
  // ADD HERE the code to stop your pipeline
  if (m_pipelineLoopTask != nullptr)
    m_pipelineLoopTask->stop();
  return FrameworkReturnCode::_SUCCESS;
}

SinkReturnCode MyPipeline::update(Transform3Df& pose)
{
  // ADD HERE the code to update the pose of the camera

  return SinkReturnCode::_NOT_IMPLEMENTED;
}

}
}
1 As we will only need SolARModuleOpenCV and SolARModuleTools for this pipeline, we add their trait header files.
2 Implementation of the pipeline loop. If an input image has been updated, we will access to it through the source component and we pass it to the sink component. If not, we read the last image captured by the camera and we pass it to the sink component.
3 We instantiate the 3 components required for this pipeline.
4 A simple code to get camera parameters and return it.
5 Set the m_externalInputImageMode to true to inform the pipeline that it run with external input images, and put the image in the buffer of the source component.
6 Set the image buffer for the sink component.
7 Start the camera.

Export your pipeline for Unity

First of all, you must follow the installation instructions of Unity and of the SolAR plugin for unity available in the section Use Unity.

Then, copy the shared binaries of your pipeline, of its modules and of the dependencies of its modules in the Assets/Plugins folder of your Unity project.

Copy the configuration file of your pipeline in the Assets/SolAR/Pipelines folder of your Unity project. When copied, edit the configuration file and replace all module paths so they are relative to your Unity project (./Assets/Plugins).

Now, in the Unity editor, you can select again your pipeline folder in the inspector of your SolARPipelineLoader, and select in the pipeline list your pipeline.

If you want to export your pipeline, create a new Unity package with:

  • The shared binaries stored in the Unity plugin folder relative to:

    • Your pipeline

    • The modules used by your pipeline

    • The third parties of the modules used by your pipeline

  • The pipleline configuration file stored under Assets/SolAR/Pipelines/.

Test your pipeline in C++

To test your pipeline in C, have a look to the section <<../use/cplusplus/index#useC, use in C++>>.