Get landmarks ofan external JPG image

[idata]

Hi,

I'm wondering if I can get the landmarks of an external image from a dataset using the SDK Intel algorithms. Or if I can convert this one to an 'Intel Image'.

I mean, I want to extract the landmarks from a JPG face but for that I think I need to convert this file into an PXCImage. I don't know if I right with this supposition, hope someone can help.

Thanks in advance

[idata]

Hi MQMQ,

Thanks for your interest in the Intel® Realsense™ Platform.

At the moment I only have knowledge of getting the landmark features from a face configuration https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_face_face_landmark_data.html Intel® RealSense™ SDK 2016 R2 Documentation . Looking at this thread , it seems that MartyG recommended you the PXCVideoModule function, please let us investigate if this function is no longer part of the Realsense API.

We appreciate your patience, have a nice day!

Best Regards,

-Jose P.

[idata]

Hi MQMQ,

We appreciate your patience.Unfortunately, extracting landmarks from a JPG image is an unsupported use case. It may or may not work, we would like encourage to try to achieve this and share your results with the community. It will be really helpful for other users looking to get the landmark from an external JPG image.Best Regards,-Jose P.

[idata]

Thank you, I haven't been able to do it yet, but if I find something, sure I post it here.

[CG_]

Hello.

I've been working towards the same functionality, and I've achieved it - to a certain point.

Currently using C# and Intel(R) RealSense(TM) SDK 10.0.26.396, and using Columbia Gaze Data Set for analysis.

The code that I wrote gets the landmarks and face detection for a single jpg image but, after that, the program always exits with one of the following errors:

• The program '[15956] DF_FaceTracking.cs.vshost.exe' has exited with code -1073740940 (0xc0000374), or
• An unhandled exception of type 'System.AccessViolationException' occurred in libpxcclr.cs.dll

This is the relevant excerpt:

public void ExternalPipeline2()

{

PXCMSenseManager pp = m_form.Session.CreateSenseManager();

if (pp == null)

{

throw new Exception("PXCMSenseManager null");

}

PXCMCaptureManager captureMgr = pp.captureManager;

if (captureMgr == null)

{

throw new Exception("PXCMCaptureManager null");

}

// Set Module

pp.EnableFace();

PXCMFaceModule faceModule = pp.QueryFace();

if (faceModule == null)

{

Debug.Assert(faceModule != null);

return;

}

//Up until here, it's just the same as the FaceTracking sample.

PXCMFaceConfiguration moduleConfiguration = faceModule.CreateActiveConfiguration();

if (moduleConfiguration == null)

{

Debug.Assert(moduleConfiguration != null);

return;

}

moduleConfiguration.SetTrackingMode(PXCMFaceConfiguration.TrackingModeType.FACE_MODE_COLOR);

moduleConfiguration.strategy = PXCMFaceConfiguration.TrackingStrategyType.STRATEGY_RIGHT_TO_LEFT;

moduleConfiguration.detection.maxTrackedFaces = 1;

moduleConfiguration.landmarks.maxTrackedFaces = 1;

moduleConfiguration.pose.maxTrackedFaces = 1;

moduleConfiguration.detection.isEnabled = true;

moduleConfiguration.landmarks.isEnabled = true;

moduleConfiguration.pose.isEnabled = false;

pxcmStatus applyChangesStatus = moduleConfiguration.ApplyChanges();

if (applyChangesStatus < pxcmStatus.PXCM_STATUS_NO_ERROR || pp.Init() < pxcmStatus.PXCM_STATUS_NO_ERROR)

{

Console.WriteLine("Init Failed");

m_form.UpdateStatus("Init Failed", MainForm.Label.StatusLabel);

}

else

{

// Read from file

string sourceDir = @"C:\Directory\"; //There's only one image in the source folder

string[] fileEntries = Directory.GetFiles(sourceDir, "*.jpg");

PXCMVideoModule videoModule = faceModule.QueryInstance();

foreach (string fileEntry in fileEntries)

{

Bitmap b = (Bitmap)ResizeImage(fileEntry); //I'm resizing it to a max of 1920x1280

w = b.Width;

h = b.Height;

PXCMImage.ImageInfo iinfo = new PXCMImage.ImageInfo()

{

width = b.Width,

height = b.Height,

format = PXCMImage.PixelFormat.PIXEL_FORMAT_RGB24 //All images I'm working with are rgb24

};

// Lock bits to mass edit

BitmapData bdata = b.LockBits(new Rectangle(0, 0, b.Width, b.Height),

ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);

// Get the address of the first line.

IntPtr ptr = bdata.Scan0;

// Declare an array to hold the bytes of the bitmap.

int bytes = b.Width * b.Height;

byte[] rgbValues = new byte[bytes];

// Copy the RGB values into the array.

System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, bytes);

PXCMImage.ImageData idata = new PXCMImage.ImageData();

idata.format = PXCMImage.PixelFormat.PIXEL_FORMAT_RGB24; //This should match the info object, I reckon

idata.planes[0] = ptr; //This is the critical bit. This is the actual data.

idata.pitches[0] = bdata.Stride; // ALIGN64(iinfo.width * 4);

// Create image with data and info

PXCMImage img = m_form.Session.CreateImage(iinfo, idata);

b.UnlockBits(bdata);

//Create sample with image

PXCMCapture.Sample sample = new PXCMCapture.Sample();

sample.color = img;

PXCMSyncPoint syn;

pxcmStatus status;

PXCMVideoModule.DataDesc inputs = new PXCMVideoModule.DataDesc();

status = videoModule.QueryCaptureProfile(0, out inputs);

// Send sample

status = videoModule.ProcessImageAsync(sample, out syn);

if (status < pxcmStatus.PXCM_STATUS_NO_ERROR)

{ // handle fail

}

else

{

// Synchronize to process image sent to VideoModule

syn.Synchronize();

using (PXCMFaceData moduleOutput = faceModule.CreateOutput())

{

// Face processing is done at this point, you can query face data

Debug.Assert(moduleOutput != null);

moduleOutput.Update();

int cf = moduleOutput.QueryNumberOfDetectedFaces();

DisplayOnBitmap(sample.color); //Same method as in FaceTracking Sample

Save(moduleOutput,...