Main MRPT website > C++ reference for MRPT 1.5.0
List of all members | Classes | Public Member Functions | Static Public Member Functions | Static Public Attributes | Protected Member Functions | Protected Attributes
mrpt::obs::CObservation3DRangeScan Class Reference

Detailed Description

Declares a class derived from "CObservation" that encapsules a 3D range scan measurement, as from a time-of-flight range camera or any other RGBD sensor.

This kind of observations can carry one or more of these data fields:

The coordinates of the 3D point cloud are in meters with respect to the depth camera origin of coordinates (in SwissRanger, the front face of the camera: a small offset ~1cm in front of the physical focal point), with the +X axis pointing forward, +Y pointing left-hand and +Z pointing up. By convention, a 3D point with its coordinates set to (0,0,0), will be considered as invalid. The field CObservation3DRangeScan::relativePoseIntensityWRTDepth describes the change of coordinates from the depth camera to the intensity (RGB or grayscale) camera. In a SwissRanger camera both cameras coincide, so this pose is just a rotation (0,0,0,-90deg,0,-90deg). But in Microsoft Kinect there is also an offset, as shown in this figure:

In any case, check the field relativePoseIntensityWRTDepth, or the method doDepthAndIntensityCamerasCoincide() to determine if both frames of reference coincide, since even for Kinect cameras both can coincide if the images have been rectified.

The 2D images and matrices are stored as common images, with an up->down rows order and left->right, as usual. Optionally, the intensity and confidence channels can be set to delayed-load images for off-rawlog storage so it saves memory by having loaded in memory just the needed images. See the methods load() and unload(). Due to the intensive storage requirements of this kind of observations, this observation is the only one in MRPT for which it's recommended to always call "load()" and "unload()" before and after using the observation, ONLY when the observation was read from a rawlog dataset, in order to make sure that all the externally stored data fields are loaded and ready in memory.

Classes that grab observations of this type are:

There are two sets of calibration parameters (see mrpt::vision::checkerBoardStereoCalibration() or the ready-to-use GUI program kinect-calibrate):

In some cameras, like SwissRanger, both are the same. It is possible in Kinect to rectify the range images such both cameras seem to coincide and then both sets of camera parameters will be identical.

Range data can be interpreted in two different ways depending on the 3D camera (this field is already set to the correct setting when grabbing observations from an mrpt::hwdrivers sensor):

The "intensity" channel may come from different channels in sesnsors as Kinect. Look at field intensityImageChannel to find out if the image was grabbed from the visible (RGB) or IR channels.

3D point clouds can be generated at any moment after grabbing with CObservation3DRangeScan::project3DPointsFromDepthImage() and CObservation3DRangeScan::project3DPointsFromDepthImageInto(), provided the correct calibration parameters. Note that project3DPointsFromDepthImage() will store the point cloud in sensor-centric local coordinates. Use project3DPointsFromDepthImageInto() to directly obtain vehicle or world coordinates.

Example of how to assign labels to pixels (for object segmentation, semantic information, etc.):

// Assume obs of type CObservation3DRangeScanPtr
obs->pixelLabels = CObservation3DRangeScan::TPixelLabelInfoPtr( new CObservation3DRangeScan::TPixelLabelInfo<NUM_BYTES>() );
obs->pixelLabels->setSize(ROWS,COLS);
obs->pixelLabels->setLabel(col,row, label_idx); // label_idxs = [0,2^NUM_BYTES-1]
//...
Note
Starting at serialization version 2 (MRPT 0.9.1+), the confidence channel is stored as an image instead of a matrix to optimize memory and disk space.
Starting at serialization version 3 (MRPT 0.9.1+), the 3D point cloud and the rangeImage can both be stored externally to save rawlog space.
Starting at serialization version 5 (MRPT 0.9.5+), the new field range_is_depth
Starting at serialization version 6 (MRPT 0.9.5+), the new field intensityImageChannel
Starting at serialization version 7 (MRPT 1.3.1+), new fields for semantic labeling
Since MRPT 1.5.0, external files format can be selected at runtime with CObservation3DRangeScan::EXTERNALS_AS_TEXT
See also
mrpt::hwdrivers::CSwissRanger3DCamera, mrpt::hwdrivers::CKinect, CObservation

Definition at line 140 of file obs/CObservation3DRangeScan.h.

#include <mrpt/obs/CObservation3DRangeScan.h>

Inheritance diagram for mrpt::obs::CObservation3DRangeScan:
Inheritance graph

Classes

struct  TCached3DProjTables
 Look-up-table struct for project3DPointsFromDepthImageInto() More...
 
struct  TPixelLabelInfo
 
struct  TPixelLabelInfoBase
 Virtual interface to all pixel-label information structs. More...
 

Public Member Functions

void * operator new (size_t size)
 
void * operator new[] (size_t size)
 
void operator delete (void *ptr) throw ()
 
void operator delete[] (void *ptr) throw ()
 
void operator delete (void *memory, void *ptr) throw ()
 
void * operator new (size_t size, const std::nothrow_t &) throw ()
 
void operator delete (void *ptr, const std::nothrow_t &) throw ()
 
 CObservation3DRangeScan ()
 Default constructor. More...
 
virtual ~CObservation3DRangeScan ()
 Destructor. More...
 
template<class POINTMAP >
void project3DPointsFromDepthImageInto (POINTMAP &dest_pointcloud, const T3DPointsProjectionParams &projectParams, const TRangeImageFilterParams &filterParams=TRangeImageFilterParams())
 Project the RGB+D images into a 3D point cloud (with color if the target map supports it) and optionally at a given 3D pose. More...
 
template<class POINTMAP >
void project3DPointsFromDepthImageInto (POINTMAP &dest_pointcloud, const bool takeIntoAccountSensorPoseOnRobot, const mrpt::poses::CPose3D *robotPoseInTheWorld=NULL, const bool PROJ3D_USE_LUT=true, const mrpt::math::CMatrix *rangeMask_min=NULL)
 
void project3DPointsFromDepthImage (const bool PROJ3D_USE_LUT=true)
 This method is equivalent to project3DPointsFromDepthImageInto() storing the projected 3D points (without color, in local sensor-centric coordinates) in this same class. More...
 
void convertTo2DScan (mrpt::obs::CObservation2DRangeScan &out_scan2d, const T3DPointsTo2DScanParams &scanParams, const TRangeImageFilterParams &filterParams=TRangeImageFilterParams())
 Convert this 3D observation into an "equivalent 2D fake laser scan", with a configurable vertical FOV. More...
 
void convertTo2DScan (mrpt::obs::CObservation2DRangeScan &out_scan2d, const std::string &sensorLabel, const double angle_sup=mrpt::utils::DEG2RAD(5), const double angle_inf=mrpt::utils::DEG2RAD(5), const double oversampling_ratio=1.2, const mrpt::math::CMatrix *rangeMask_min=NULL)
 
size_t getScanSize () const
 Get the size of the scan pointcloud. More...
 
void getDescriptionAsText (std::ostream &o) const MRPT_OVERRIDE
 Build a detailed, multi-line textual description of the observation contents and dump it to the output stream. More...
 
void swap (CObservation3DRangeScan &o)
 Very efficient method to swap the contents of two observations. More...
 
void getZoneAsObs (CObservation3DRangeScan &obs, const unsigned int &r1, const unsigned int &r2, const unsigned int &c1, const unsigned int &c2)
 Extract a ROI of the 3D observation as a new one. More...
 
template<class METRICMAP >
bool insertObservationInto (METRICMAP *theMap, const mrpt::poses::CPose3D *robotPose=NULL) const
 This method is equivalent to: More...
 
void getSensorPose (mrpt::math::TPose3D &out_sensorPose) const
 A general method to retrieve the sensor pose on the robot. More...
 
void setSensorPose (const mrpt::math::TPose3D &newSensorPose)
 A general method to change the sensor pose on the robot. More...
 
virtual mxArraywriteToMatlab () const
 Introduces a pure virtual method responsible for writing to a mxArray Matlab object, typically a MATLAB struct whose contents are documented in each derived class. More...
 
mrpt::utils::CObjectPtr duplicateGetSmartPtr () const
 Returns a copy of the object, indepently of its class, as a smart pointer (the newly created object will exist as long as any copy of this smart pointer). More...
 
CObject * clone () const
 Cloning interface for smart pointers. More...
 
Delayed-load manual control methods.
virtual void load () const MRPT_OVERRIDE
 Makes sure all images and other fields which may be externally stored are loaded in memory. More...
 
virtual void unload () MRPT_OVERRIDE
 Unload all images, for the case they being delayed-load images stored in external files (othewise, has no effect). More...
 
Point cloud external storage functions
bool points3D_isExternallyStored () const
 
std::string points3D_getExternalStorageFile () const
 
void points3D_getExternalStorageFileAbsolutePath (std::string &out_path) const
 
std::string points3D_getExternalStorageFileAbsolutePath () const
 
void points3D_convertToExternalStorage (const std::string &fileName, const std::string &use_this_base_dir)
 Users won't normally want to call this, it's only used from internal MRPT programs. More...
 
Range Matrix external storage functions
bool rangeImage_isExternallyStored () const
 
std::string rangeImage_getExternalStorageFile () const
 
void rangeImage_getExternalStorageFileAbsolutePath (std::string &out_path) const
 
std::string rangeImage_getExternalStorageFileAbsolutePath () const
 
void rangeImage_convertToExternalStorage (const std::string &fileName, const std::string &use_this_base_dir)
 Users won't normally want to call this, it's only used from internal MRPT programs. More...
 
void rangeImage_forceResetExternalStorage ()
 Forces marking this observation as non-externally stored - it doesn't anything else apart from reseting the corresponding flag (Users won't normally want to call this, it's only used from internal MRPT programs) More...
 

Static Public Member Functions

static void * operator new (size_t size, void *ptr)
 
static double recoverCameraCalibrationParameters (const CObservation3DRangeScan &in_obs, mrpt::utils::TCamera &out_camParams, const double camera_offset=0.01)
 A Levenberg-Marquart-based optimizer to recover the calibration parameters of a 3D camera given a range (depth) image and the corresponding 3D point cloud. More...
 

Public Attributes

Confidence "channel"
bool hasConfidenceImage
 true means the field confidenceImage contains valid data More...
 
mrpt::utils::CImage confidenceImage
 If hasConfidenceImage=true, an image with the "confidence" value [range 0-255] as estimated by the capture drivers. More...
 

Static Public Attributes

static bool EXTERNALS_AS_TEXT
 Whether external files (3D points, range and confidence) are to be saved as .txt text files (MATLAB compatible) or *.bin binary (faster). More...
 
static TCached3DProjTables m_3dproj_lut
 3D point cloud projection look-up-table More...
 
static const mrpt::utils::TRuntimeClassId classCObject
 
RTTI stuff
static const mrpt::utils::TRuntimeClassId classCObservation
 
RTTI stuff
static const mrpt::utils::TRuntimeClassId classCSerializable
 

Protected Member Functions

void swap (CObservation &o)
 Swap with another observation, ONLY the data defined here in the base class CObservation. It's protected since it'll be only called from child classes that should know what else to swap appart from these common data. More...
 
CSerializable virtual methods
void writeToStream (mrpt::utils::CStream &out, int *getVersion) const
 Introduces a pure virtual method responsible for writing to a CStream. More...
 
void readFromStream (mrpt::utils::CStream &in, int version)
 Introduces a pure virtual method responsible for loading from a CStream This can not be used directly be users, instead use "stream >> object;" for reading it from a stream or "stream >> object_ptr;" if the class is unknown apriori. More...
 

Protected Attributes

bool m_points3D_external_stored
 If set to true, m_points3D_external_file is valid. More...
 
std::string m_points3D_external_file
 3D points are in CImage::IMAGES_PATH_BASE+<this_file_name> More...
 
bool m_rangeImage_external_stored
 If set to true, m_rangeImage_external_file is valid. More...
 
std::string m_rangeImage_external_file
 rangeImage is in CImage::IMAGES_PATH_BASE+<this_file_name> More...
 

Data common to any observation

mrpt::system::TTimeStamp timestamp
 The associated UTC time-stamp. Where available, this should contain the accurate satellite-based timestamp of the sensor reading. More...
 
std::string sensorLabel
 An arbitrary label that can be used to identify the sensor. More...
 
mrpt::system::TTimeStamp getTimeStamp () const
 Returns CObservation::timestamp for all kind of observations. More...
 
virtual mrpt::system::TTimeStamp getOriginalReceivedTimeStamp () const
 By default, returns CObservation::timestamp but in sensors capable of satellite (or otherwise) accurate UTC timing of readings, this contains the computer-based timestamp of reception, which may be slightly different than timestamp. More...
 

RTTI stuff

typedef CObservation3DRangeScanPtr SmartPtr
 
static mrpt::utils::CLASSINIT _init_CObservation3DRangeScan
 
static mrpt::utils::TRuntimeClassId classCObservation3DRangeScan
 
static const mrpt::utils::TRuntimeClassIdclassinfo
 
static const mrpt::utils::TRuntimeClassId_GetBaseClass ()
 
virtual const mrpt::utils::TRuntimeClassIdGetRuntimeClass () const
 Returns information about the class of an object in runtime. More...
 
virtual mrpt::utils::CObjectduplicate () const
 Returns a copy of the object, indepently of its class. More...
 
static mrpt::utils::CObjectCreateObject ()
 
static CObservation3DRangeScanPtr Create ()
 

Point cloud

bool hasPoints3D
 true means the field points3D contains valid data. More...
 
std::vector< float > points3D_x
 
std::vector< float > points3D_y
 
std::vector< float > points3D_z
 If hasPoints3D=true, the (X,Y,Z) coordinates of the 3D point cloud detected by the camera. More...
 
std::vector< uint16_t > points3D_idxs_x
 
std::vector< uint16_t > points3D_idxs_y
 //!< If hasPoints3D=true, the (x,y) pixel coordinates for each (X,Y,Z) point in points3D_x, points3D_y, points3D_z More...
 
void resizePoints3DVectors (const size_t nPoints)
 Use this method instead of resizing all three points3D_x, points3D_y & points3D_z to allow the usage of the internal memory pool. More...
 

Range (depth) image

bool hasRangeImage
 true means the field rangeImage contains valid data More...
 
mrpt::math::CMatrix rangeImage
 If hasRangeImage=true, a matrix of floats with the range data as captured by the camera (in meters) More...
 
bool range_is_depth
 true: Kinect-like ranges: entries of rangeImage are distances along the +X axis; false: Ranges in rangeImage are actual distances in 3D. More...
 
void rangeImage_setSize (const int HEIGHT, const int WIDTH)
 Similar to calling "rangeImage.setSize(H,W)" but this method provides memory pooling to speed-up the memory allocation. More...
 

Intensity (RGB) channels

enum  TIntensityChannelID { CH_VISIBLE = 0, CH_IR = 1 }
 Enum type for intensityImageChannel. More...
 
bool hasIntensityImage
 true means the field intensityImage contains valid data More...
 
mrpt::utils::CImage intensityImage
 If hasIntensityImage=true, a color or gray-level intensity image of the same size than "rangeImage". More...
 
TIntensityChannelID intensityImageChannel
 The source of the intensityImage; typically the visible channel. More...
 

Pixel-wise classification labels (for semantic labeling, etc.)

typedef stlplus::smart_ptr< TPixelLabelInfoBaseTPixelLabelInfoPtr
 Used in CObservation3DRangeScan::pixelLabels. More...
 
TPixelLabelInfoPtr pixelLabels
 All information about pixel labeling is stored in this (smart pointer to) structure; refer to TPixelLabelInfo for details on the contents User is responsible of creating a new object of the desired data type. More...
 
bool hasPixelLabels () const
 Returns true if the field CObservation3DRangeScan::pixelLabels contains a non-NULL smart pointer. More...
 

Sensor parameters

mrpt::utils::TCamera cameraParams
 Projection parameters of the depth camera. More...
 
mrpt::utils::TCamera cameraParamsIntensity
 Projection parameters of the intensity (graylevel or RGB) camera. More...
 
mrpt::poses::CPose3D relativePoseIntensityWRTDepth
 Relative pose of the intensity camera wrt the depth camera (which is the coordinates origin for this observation). More...
 
float maxRange
 The maximum range allowed by the device, in meters (e.g. 8.0m, 5.0m,...) More...
 
mrpt::poses::CPose3D sensorPose
 The 6D pose of the sensor on the robot. More...
 
float stdError
 The "sigma" error of the device in meters, used while inserting the scan in an occupancy grid. More...
 
bool doDepthAndIntensityCamerasCoincide () const
 Return true if relativePoseIntensityWRTDepth equals the pure rotation (0,0,0,-90deg,0,-90deg) (with a small comparison epsilon) More...
 
void getSensorPose (mrpt::poses::CPose3D &out_sensorPose) const MRPT_OVERRIDE
 A general method to retrieve the sensor pose on the robot. More...
 
void setSensorPose (const mrpt::poses::CPose3D &newSensorPose) MRPT_OVERRIDE
 A general method to change the sensor pose on the robot. More...
 

Member Typedef Documentation

typedef CObservation3DRangeScanPtr mrpt::obs::CObservation3DRangeScan::SmartPtr

A typedef for the associated smart pointer

Definition at line 143 of file obs/CObservation3DRangeScan.h.

Member Enumeration Documentation

Enum type for intensityImageChannel.

Enumerator
CH_VISIBLE 

Grayscale or RGB visible channel of the camera sensor.

CH_IR 

Infrarred (IR) channel.

Definition at line 342 of file obs/CObservation3DRangeScan.h.

Constructor & Destructor Documentation

mrpt::obs::CObservation3DRangeScan::CObservation3DRangeScan ( )

Default constructor.

virtual mrpt::obs::CObservation3DRangeScan::~CObservation3DRangeScan ( )
virtual

Destructor.

Member Function Documentation

static const mrpt::utils::TRuntimeClassId* mrpt::obs::CObservation3DRangeScan::_GetBaseClass ( )
staticprotected
CObject* mrpt::utils::CObject::clone ( ) const
inlineinherited

Cloning interface for smart pointers.

Definition at line 143 of file CObject.h.

void mrpt::obs::CObservation3DRangeScan::convertTo2DScan ( mrpt::obs::CObservation2DRangeScan out_scan2d,
const T3DPointsTo2DScanParams scanParams,
const TRangeImageFilterParams filterParams = TRangeImageFilterParams() 
)

Convert this 3D observation into an "equivalent 2D fake laser scan", with a configurable vertical FOV.

The result is a 2D laser scan with more "rays" (N) than columns has the 3D observation (W), exactly: N = W * oversampling_ratio. This oversampling is required since laser scans sample the space at evenly-separated angles, while a range camera follows a tangent-like distribution. By oversampling we make sure we don't leave "gaps" unseen by the virtual "2D laser".

All obstacles within a frustum are considered and the minimum distance is kept in each direction. The horizontal FOV of the frustum is automatically computed from the intrinsic parameters, but the vertical FOV must be provided by the user, and can be set to be assymetric which may be useful depending on the zone of interest where to look for obstacles.

All spatial transformations are riguorosly taken into account in this class, using the depth camera intrinsic calibration parameters.

The timestamp of the new object is copied from the 3D object. Obviously, a requisite for calling this method is the 3D observation having range data, i.e. hasRangeImage must be true. It's not needed to have RGB data nor the raw 3D point clouds for this method to work.

If scanParams.use_origin_sensor_pose is true, the points will be projected to 3D and then reprojected as seen from a different sensorPose at the vehicle frame origin. Otherwise (the default), the output 2D observation will share the sensorPose of the input 3D scan (using a more efficient algorithm that avoids trigonometric functions).

Parameters
[out]out_scan2dThe resulting 2D equivalent scan.
See also
The example in http://www.mrpt.org/tutorials/mrpt-examples/example-kinect-to-2d-laser-demo/
void mrpt::obs::CObservation3DRangeScan::convertTo2DScan ( mrpt::obs::CObservation2DRangeScan out_scan2d,
const std::string &  sensorLabel,
const double  angle_sup = mrpt::utils::DEG2RAD(5),
const double  angle_inf = mrpt::utils::DEG2RAD(5),
const double  oversampling_ratio = 1.2,
const mrpt::math::CMatrix rangeMask_min = NULL 
)
static CObservation3DRangeScanPtr mrpt::obs::CObservation3DRangeScan::Create ( )
static
static mrpt::utils::CObject* mrpt::obs::CObservation3DRangeScan::CreateObject ( )
static
bool mrpt::obs::CObservation3DRangeScan::doDepthAndIntensityCamerasCoincide ( ) const

Return true if relativePoseIntensityWRTDepth equals the pure rotation (0,0,0,-90deg,0,-90deg) (with a small comparison epsilon)

See also
relativePoseIntensityWRTDepth

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

virtual mrpt::utils::CObject* mrpt::obs::CObservation3DRangeScan::duplicate ( ) const
virtual

Returns a copy of the object, indepently of its class.

Implements mrpt::utils::CObject.

mrpt::utils::CObjectPtr mrpt::utils::CObject::duplicateGetSmartPtr ( ) const
inlineinherited

Returns a copy of the object, indepently of its class, as a smart pointer (the newly created object will exist as long as any copy of this smart pointer).

Definition at line 140 of file CObject.h.

void mrpt::obs::CObservation3DRangeScan::getDescriptionAsText ( std::ostream &  o) const
virtual

Build a detailed, multi-line textual description of the observation contents and dump it to the output stream.

Note
If overried by derived classes, call base CObservation::getDescriptionAsText() first to show common information.
This is the text that appears in RawLogViewer when selecting an object in the dataset

Reimplemented from mrpt::obs::CObservation.

virtual mrpt::system::TTimeStamp mrpt::obs::CObservation::getOriginalReceivedTimeStamp ( ) const
inlinevirtualinherited

By default, returns CObservation::timestamp but in sensors capable of satellite (or otherwise) accurate UTC timing of readings, this contains the computer-based timestamp of reception, which may be slightly different than timestamp.

See also
getTimeStamp()

Reimplemented in mrpt::obs::CObservationGPS, and mrpt::obs::CObservationVelodyneScan.

Definition at line 58 of file obs/CObservation.h.

virtual const mrpt::utils::TRuntimeClassId* mrpt::obs::CObservation3DRangeScan::GetRuntimeClass ( ) const
virtual

Returns information about the class of an object in runtime.

Reimplemented from mrpt::obs::CObservation.

size_t mrpt::obs::CObservation3DRangeScan::getScanSize ( ) const

Get the size of the scan pointcloud.

Note
Method is added for compatibility with its CObservation2DRangeScan counterpart
void mrpt::obs::CObservation::getSensorPose ( mrpt::math::TPose3D out_sensorPose) const
inherited

A general method to retrieve the sensor pose on the robot.

Note that most sensors will return a full (6D) CPose3D, but see the derived classes for more details or special cases.

See also
setSensorPose
void mrpt::obs::CObservation3DRangeScan::getSensorPose ( mrpt::poses::CPose3D out_sensorPose) const
inlinevirtual

A general method to retrieve the sensor pose on the robot.

Note that most sensors will return a full (6D) CPose3D, but see the derived classes for more details or special cases.

See also
setSensorPose

Implements mrpt::obs::CObservation.

Definition at line 551 of file obs/CObservation3DRangeScan.h.

mrpt::system::TTimeStamp mrpt::obs::CObservation::getTimeStamp ( ) const
inlineinherited

Returns CObservation::timestamp for all kind of observations.

See also
getOriginalReceivedTimeStamp()

Definition at line 56 of file obs/CObservation.h.

void mrpt::obs::CObservation3DRangeScan::getZoneAsObs ( CObservation3DRangeScan obs,
const unsigned int &  r1,
const unsigned int &  r2,
const unsigned int &  c1,
const unsigned int &  c2 
)

Extract a ROI of the 3D observation as a new one.

Note
PixelLabels are not copied to the output subimage.
bool mrpt::obs::CObservation3DRangeScan::hasPixelLabels ( ) const
inline

Returns true if the field CObservation3DRangeScan::pixelLabels contains a non-NULL smart pointer.

To enhance a 3D point cloud with labeling info, just assign an appropiate object to pixelLabels

Definition at line 364 of file obs/CObservation3DRangeScan.h.

template<class METRICMAP >
bool mrpt::obs::CObservation::insertObservationInto ( METRICMAP *  theMap,
const mrpt::poses::CPose3D robotPose = NULL 
) const
inlineinherited

This method is equivalent to:

map->insertObservation(this, robotPose)
Parameters
theMapThe map where this observation is to be inserted: the map will be updated.
robotPoseThe pose of the robot base for this observation, relative to the target metric map. Set to NULL (default) to use (0,0,0deg)
Returns
Returns true if the map has been updated, or false if this observations has nothing to do with a metric map (for example, a sound observation).

See: Maps and observations compatibility matrix

See also
CMetricMap, CMetricMap::insertObservation

Definition at line 77 of file obs/CObservation.h.

virtual void mrpt::obs::CObservation3DRangeScan::load ( ) const
virtual

Makes sure all images and other fields which may be externally stored are loaded in memory.

Note that for all CImages, calling load() is not required since the images will be automatically loaded upon first access, so load() shouldn't be needed to be called in normal cases by the user. If all the data were alredy loaded or this object has no externally stored data fields, calling this method has no effects.

See also
unload

Reimplemented from mrpt::obs::CObservation.

Referenced by mrpt::maps::COctoMapBase< OCTREE, OCTREE_NODE >::internal_build_PointCloud_for_observation().

void mrpt::obs::CObservation3DRangeScan::operator delete ( void *  ptr)
throw (
)
inline

Definition at line 143 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::operator delete ( void *  memory,
void *  ptr 
)
throw (
)
inline

Definition at line 143 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::operator delete ( void *  ptr,
const std::nothrow_t &   
)
throw (
)
inline

Definition at line 143 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::operator delete[] ( void *  ptr)
throw (
)
inline

Definition at line 143 of file obs/CObservation3DRangeScan.h.

static void* mrpt::obs::CObservation3DRangeScan::operator new ( size_t  size,
void *  ptr 
)
inlinestatic

Definition at line 143 of file obs/CObservation3DRangeScan.h.

void* mrpt::obs::CObservation3DRangeScan::operator new ( size_t  size,
const std::nothrow_t &   
)
throw (
)
inline

Definition at line 143 of file obs/CObservation3DRangeScan.h.

void* mrpt::obs::CObservation3DRangeScan::operator new ( size_t  size)
inline

Definition at line 143 of file obs/CObservation3DRangeScan.h.

void* mrpt::obs::CObservation3DRangeScan::operator new[] ( size_t  size)
inline

Definition at line 143 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::points3D_convertToExternalStorage ( const std::string &  fileName,
const std::string &  use_this_base_dir 
)

Users won't normally want to call this, it's only used from internal MRPT programs.

See also
EXTERNALS_AS_TEXT
std::string mrpt::obs::CObservation3DRangeScan::points3D_getExternalStorageFile ( ) const
inline

Definition at line 304 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::points3D_getExternalStorageFileAbsolutePath ( std::string &  out_path) const
std::string mrpt::obs::CObservation3DRangeScan::points3D_getExternalStorageFileAbsolutePath ( ) const
inline

Definition at line 306 of file obs/CObservation3DRangeScan.h.

bool mrpt::obs::CObservation3DRangeScan::points3D_isExternallyStored ( ) const
inline

Definition at line 303 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::project3DPointsFromDepthImage ( const bool  PROJ3D_USE_LUT = true)
inline

This method is equivalent to project3DPointsFromDepthImageInto() storing the projected 3D points (without color, in local sensor-centric coordinates) in this same class.

For new code it's recommended to use instead project3DPointsFromDepthImageInto() which is much more versatile.

Definition at line 232 of file obs/CObservation3DRangeScan.h.

References mrpt::mrpt::utils::DEG2RAD(), MRPT_DEPRECATED, mrpt::obs::T3DPointsProjectionParams::PROJ3D_USE_LUT, mrpt::obs::detail::project3DPointsFromDepthImageInto(), and mrpt::obs::T3DPointsProjectionParams::takeIntoAccountSensorPoseOnRobot.

template<class POINTMAP >
void mrpt::obs::CObservation3DRangeScan::project3DPointsFromDepthImageInto ( POINTMAP &  dest_pointcloud,
const T3DPointsProjectionParams projectParams,
const TRangeImageFilterParams filterParams = TRangeImageFilterParams() 
)
inline

Project the RGB+D images into a 3D point cloud (with color if the target map supports it) and optionally at a given 3D pose.

The 3D point coordinates are computed from the depth image (rangeImage) and the depth camera camera parameters (cameraParams). There exist two set of formulas for projecting the i'th point, depending on the value of "range_is_depth". In all formulas below, "rangeImage" is the matrix of ranges and the pixel coordinates are (r,c).

1) [range_is_depth=true] With "range equals depth" or "Kinect-like depth mode": the range values are in fact distances along the "+X" axis, not real 3D ranges (this is the way Kinect reports ranges):

x(i) = rangeImage(r,c)
y(i) = (r_cx - c) * x(i) / r_fx
z(i) = (r_cy - r) * x(i) / r_fy

2) [range_is_depth=false] With "normal ranges": range means distance in 3D. This must be set when processing data from the SwissRange 3D camera, among others.

Ky = (r_cx - c)/r_fx
Kz = (r_cy - r)/r_fy
x(i) = rangeImage(r,c) / sqrt( 1 + Ky^2 + Kz^2 )
y(i) = Ky * x(i)
z(i) = Kz * x(i)

The color of each point is determined by projecting the 3D local point into the RGB image using cameraParamsIntensity.

By default the local (sensor-centric) coordinates of points are directly stored into the local map, but if indicated so in takeIntoAccountSensorPoseOnRobot the points are transformed with sensorPose. Furthermore, if provided, those coordinates are transformed with robotPoseInTheWorld

Template Parameters
POINTMAPSupported maps are all those covered by mrpt::utils::PointCloudAdapter (mrpt::maps::CPointsMap and derived, mrpt::opengl::CPointCloudColoured, PCL point clouds,...)
Note
In MRPT < 0.9.5, this method always assumes that ranges were in Kinect-like format.

Definition at line 207 of file obs/CObservation3DRangeScan.h.

References MRPT_DEPRECATED.

template<class POINTMAP >
void mrpt::obs::CObservation3DRangeScan::project3DPointsFromDepthImageInto ( POINTMAP &  dest_pointcloud,
const bool  takeIntoAccountSensorPoseOnRobot,
const mrpt::poses::CPose3D robotPoseInTheWorld = NULL,
const bool  PROJ3D_USE_LUT = true,
const mrpt::math::CMatrix rangeMask_min = NULL 
)
inline
void mrpt::obs::CObservation3DRangeScan::rangeImage_convertToExternalStorage ( const std::string &  fileName,
const std::string &  use_this_base_dir 
)

Users won't normally want to call this, it's only used from internal MRPT programs.

See also
EXTERNALS_AS_TEXT
void mrpt::obs::CObservation3DRangeScan::rangeImage_forceResetExternalStorage ( )
inline

Forces marking this observation as non-externally stored - it doesn't anything else apart from reseting the corresponding flag (Users won't normally want to call this, it's only used from internal MRPT programs)

Definition at line 335 of file obs/CObservation3DRangeScan.h.

std::string mrpt::obs::CObservation3DRangeScan::rangeImage_getExternalStorageFile ( ) const
inline

Definition at line 326 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::rangeImage_getExternalStorageFileAbsolutePath ( std::string &  out_path) const
std::string mrpt::obs::CObservation3DRangeScan::rangeImage_getExternalStorageFileAbsolutePath ( ) const
inline

Definition at line 328 of file obs/CObservation3DRangeScan.h.

bool mrpt::obs::CObservation3DRangeScan::rangeImage_isExternallyStored ( ) const
inline

Definition at line 325 of file obs/CObservation3DRangeScan.h.

void mrpt::obs::CObservation3DRangeScan::rangeImage_setSize ( const int  HEIGHT,
const int  WIDTH 
)

Similar to calling "rangeImage.setSize(H,W)" but this method provides memory pooling to speed-up the memory allocation.

void mrpt::obs::CObservation3DRangeScan::readFromStream ( mrpt::utils::CStream in,
int  version 
)
protectedvirtual

Introduces a pure virtual method responsible for loading from a CStream This can not be used directly be users, instead use "stream >> object;" for reading it from a stream or "stream >> object_ptr;" if the class is unknown apriori.

Parameters
inThe input binary stream where the object data must read from.
versionThe version of the object stored in the stream: use this version number in your code to know how to read the incoming data.
Exceptions
std::exceptionOn any error, see CStream::ReadBuffer
See also
CStream

Implements mrpt::utils::CSerializable.

static double mrpt::obs::CObservation3DRangeScan::recoverCameraCalibrationParameters ( const CObservation3DRangeScan in_obs,
mrpt::utils::TCamera out_camParams,
const double  camera_offset = 0.01 
)
static

A Levenberg-Marquart-based optimizer to recover the calibration parameters of a 3D camera given a range (depth) image and the corresponding 3D point cloud.

Parameters
camera_offsetThe offset (in meters) in the +X direction of the point cloud. It's 1cm for SwissRanger SR4000.
Returns
The final average reprojection error per pixel (typ <0.05 px)
void mrpt::obs::CObservation3DRangeScan::resizePoints3DVectors ( const size_t  nPoints)

Use this method instead of resizing all three points3D_x, points3D_y & points3D_z to allow the usage of the internal memory pool.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto(), and mrpt::utils::PointCloudAdapter< mrpt::obs::CObservation3DRangeScan >::resize().

void mrpt::obs::CObservation::setSensorPose ( const mrpt::math::TPose3D newSensorPose)
inherited

A general method to change the sensor pose on the robot.

Note that most sensors will use the full (6D) CPose3D, but see the derived classes for more details or special cases.

See also
getSensorPose
void mrpt::obs::CObservation3DRangeScan::setSensorPose ( const mrpt::poses::CPose3D newSensorPose)
inlinevirtual

A general method to change the sensor pose on the robot.

Note that most sensors will use the full (6D) CPose3D, but see the derived classes for more details or special cases.

See also
getSensorPose

Implements mrpt::obs::CObservation.

Definition at line 553 of file obs/CObservation3DRangeScan.h.

References MRPT_OVERRIDE.

void mrpt::obs::CObservation::swap ( CObservation o)
protectedinherited

Swap with another observation, ONLY the data defined here in the base class CObservation. It's protected since it'll be only called from child classes that should know what else to swap appart from these common data.

void mrpt::obs::CObservation3DRangeScan::swap ( CObservation3DRangeScan o)

Very efficient method to swap the contents of two observations.

virtual void mrpt::obs::CObservation3DRangeScan::unload ( )
virtual

Unload all images, for the case they being delayed-load images stored in external files (othewise, has no effect).

See also
load

Reimplemented from mrpt::obs::CObservation.

virtual mxArray* mrpt::utils::CSerializable::writeToMatlab ( ) const
inlinevirtualinherited

Introduces a pure virtual method responsible for writing to a mxArray Matlab object, typically a MATLAB struct whose contents are documented in each derived class.

Returns
A new mxArray (caller is responsible of memory freeing) or NULL is class does not support conversion to MATLAB.

Definition at line 79 of file CSerializable.h.

References BASE_IMPEXP, DEFINE_MRPT_OBJECT_POST, mrpt::utils::ObjectToOctetVector(), mrpt::utils::ObjectToRawString(), mrpt::utils::ObjectToString(), mrpt::utils::OctetVectorToObject(), mrpt::utils::RawStringToObject(), and mrpt::utils::StringToObject().

void mrpt::obs::CObservation3DRangeScan::writeToStream ( mrpt::utils::CStream out,
int *  getVersion 
) const
protectedvirtual

Introduces a pure virtual method responsible for writing to a CStream.

This can not be used directly be users, instead use "stream << object;" for writing it to a stream.

Parameters
outThe output binary stream where object must be dumped.
getVersionIf NULL, the object must be dumped. If not, only the version of the object dump must be returned in this pointer. This enables the versioning of objects dumping and backward compatibility with previously stored data.
Exceptions
std::exceptionOn any error, see CStream::WriteBuffer
See also
CStream

Implements mrpt::utils::CSerializable.

Member Data Documentation

mrpt::utils::CLASSINIT mrpt::obs::CObservation3DRangeScan::_init_CObservation3DRangeScan
staticprotected

Definition at line 143 of file obs/CObservation3DRangeScan.h.

mrpt::utils::TCamera mrpt::obs::CObservation3DRangeScan::cameraParams

Projection parameters of the depth camera.

Definition at line 531 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

mrpt::utils::TCamera mrpt::obs::CObservation3DRangeScan::cameraParamsIntensity

Projection parameters of the intensity (graylevel or RGB) camera.

Definition at line 532 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

const mrpt::utils::TRuntimeClassId mrpt::utils::CObject::classCObject
staticinherited

Definition at line 128 of file CObject.h.

const mrpt::utils::TRuntimeClassId mrpt::obs::CObservation::classCObservation
staticinherited

Definition at line 43 of file obs/CObservation.h.

mrpt::utils::TRuntimeClassId mrpt::obs::CObservation3DRangeScan::classCObservation3DRangeScan
static

Definition at line 143 of file obs/CObservation3DRangeScan.h.

const mrpt::utils::TRuntimeClassId mrpt::utils::CSerializable::classCSerializable
staticinherited

Definition at line 42 of file CSerializable.h.

const mrpt::utils::TRuntimeClassId* mrpt::obs::CObservation3DRangeScan::classinfo
static

Definition at line 143 of file obs/CObservation3DRangeScan.h.

mrpt::utils::CImage mrpt::obs::CObservation3DRangeScan::confidenceImage

If hasConfidenceImage=true, an image with the "confidence" value [range 0-255] as estimated by the capture drivers.

Definition at line 356 of file obs/CObservation3DRangeScan.h.

bool mrpt::obs::CObservation3DRangeScan::EXTERNALS_AS_TEXT
static

Whether external files (3D points, range and confidence) are to be saved as .txt text files (MATLAB compatible) or *.bin binary (faster).

Loading always will determine the type by inspecting the file extension.

Note
Default=false

Definition at line 283 of file obs/CObservation3DRangeScan.h.

bool mrpt::obs::CObservation3DRangeScan::hasConfidenceImage

true means the field confidenceImage contains valid data

Definition at line 355 of file obs/CObservation3DRangeScan.h.

bool mrpt::obs::CObservation3DRangeScan::hasIntensityImage

true means the field intensityImage contains valid data

Definition at line 348 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

bool mrpt::obs::CObservation3DRangeScan::hasPoints3D
bool mrpt::obs::CObservation3DRangeScan::hasRangeImage

true means the field rangeImage contains valid data

Definition at line 316 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::graphslam::deciders::CRangeScanOps< GRAPH_T >::_getICPEdge(), and mrpt::obs::detail::project3DPointsFromDepthImageInto().

mrpt::utils::CImage mrpt::obs::CObservation3DRangeScan::intensityImage

If hasIntensityImage=true, a color or gray-level intensity image of the same size than "rangeImage".

Definition at line 349 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

TIntensityChannelID mrpt::obs::CObservation3DRangeScan::intensityImageChannel

The source of the intensityImage; typically the visible channel.

See also
TIntensityChannelID

Definition at line 350 of file obs/CObservation3DRangeScan.h.

TCached3DProjTables mrpt::obs::CObservation3DRangeScan::m_3dproj_lut
static

3D point cloud projection look-up-table

See also
project3DPointsFromDepthImage

Definition at line 579 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

std::string mrpt::obs::CObservation3DRangeScan::m_points3D_external_file
protected

3D points are in CImage::IMAGES_PATH_BASE+<this_file_name>

Definition at line 147 of file obs/CObservation3DRangeScan.h.

bool mrpt::obs::CObservation3DRangeScan::m_points3D_external_stored
protected

If set to true, m_points3D_external_file is valid.

Definition at line 146 of file obs/CObservation3DRangeScan.h.

std::string mrpt::obs::CObservation3DRangeScan::m_rangeImage_external_file
protected

rangeImage is in CImage::IMAGES_PATH_BASE+<this_file_name>

Definition at line 150 of file obs/CObservation3DRangeScan.h.

bool mrpt::obs::CObservation3DRangeScan::m_rangeImage_external_stored
protected

If set to true, m_rangeImage_external_file is valid.

Definition at line 149 of file obs/CObservation3DRangeScan.h.

float mrpt::obs::CObservation3DRangeScan::maxRange

The maximum range allowed by the device, in meters (e.g. 8.0m, 5.0m,...)

Definition at line 546 of file obs/CObservation3DRangeScan.h.

TPixelLabelInfoPtr mrpt::obs::CObservation3DRangeScan::pixelLabels

All information about pixel labeling is stored in this (smart pointer to) structure; refer to TPixelLabelInfo for details on the contents User is responsible of creating a new object of the desired data type.

It will be automatically (de)serialized no matter its specific type.

Definition at line 525 of file obs/CObservation3DRangeScan.h.

std::vector<uint16_t> mrpt::obs::CObservation3DRangeScan::points3D_idxs_x
std::vector<uint16_t> mrpt::obs::CObservation3DRangeScan::points3D_idxs_y

//!< If hasPoints3D=true, the (x,y) pixel coordinates for each (X,Y,Z) point in points3D_x, points3D_y, points3D_z

Definition at line 289 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

std::vector<float> mrpt::obs::CObservation3DRangeScan::points3D_x
std::vector<float> mrpt::obs::CObservation3DRangeScan::points3D_y
std::vector<float> mrpt::obs::CObservation3DRangeScan::points3D_z
bool mrpt::obs::CObservation3DRangeScan::range_is_depth

true: Kinect-like ranges: entries of rangeImage are distances along the +X axis; false: Ranges in rangeImage are actual distances in 3D.

Definition at line 318 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

mrpt::math::CMatrix mrpt::obs::CObservation3DRangeScan::rangeImage

If hasRangeImage=true, a matrix of floats with the range data as captured by the camera (in meters)

See also
range_is_depth

Definition at line 317 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

mrpt::poses::CPose3D mrpt::obs::CObservation3DRangeScan::relativePoseIntensityWRTDepth

Relative pose of the intensity camera wrt the depth camera (which is the coordinates origin for this observation).

In a SwissRanger camera, this will be (0,0,0,-90deg,0,-90deg) since both cameras coincide. In a Kinect, this will include a small lateral displacement and a rotation, according to the drawing on the top of this page.

See also
doDepthAndIntensityCamerasCoincide

Definition at line 539 of file obs/CObservation3DRangeScan.h.

Referenced by mrpt::obs::detail::project3DPointsFromDepthImageInto().

std::string mrpt::obs::CObservation::sensorLabel
inherited

An arbitrary label that can be used to identify the sensor.

Definition at line 53 of file obs/CObservation.h.

mrpt::poses::CPose3D mrpt::obs::CObservation3DRangeScan::sensorPose
float mrpt::obs::CObservation3DRangeScan::stdError

The "sigma" error of the device in meters, used while inserting the scan in an occupancy grid.

Definition at line 548 of file obs/CObservation3DRangeScan.h.

mrpt::system::TTimeStamp mrpt::obs::CObservation::timestamp
inherited

The associated UTC time-stamp. Where available, this should contain the accurate satellite-based timestamp of the sensor reading.

See also
getOriginalReceivedTimeStamp(), getTimeStamp()

Definition at line 52 of file obs/CObservation.h.




Page generated by Doxygen 1.8.11 for MRPT 1.5.0 Git: 4bfb26d Sun Jun 25 22:39:11 2017 +0200 at lun jun 26 00:00:32 CEST 2017