MorphCast SDK FAQ

Can the SDK identify people?

The MorphCast SDK has been specifically designed and developed to NOT identify people. Therefore, it can detect a face’s presence, motion, emotions, age and gender but not identify a person.

Which emotions can the MorphCast SDK recognize?

The SDK can identify any combination of the six basic emotions (Anger, Disgust, Fear, Happiness, Sadness, Surprise) plus a neutral emotional state.  The SDK can not only detect the presence of an emotion, but also estimate its intensity.

Do people open the camera?

Our solution, based on our metrics and our clients’ feedback, achieves camera open rates ranging from 63% to 88%, depending on the context and the content/creative.

Privacy is guaranteed by the fact the SDK doesn’t access any personal data and doesn’t store (neither locally or on a server) any images or video frames of the user’s face. For more background on this, read https://www.morphcast.com/gdpr/

Why a client-side approach?

One of the primary reasons why we decided to take a client-side approach is user privacy.

Cloud computing services, including cloud-based machine vision APIs require that video frames and images are sent to a server to be analyzed in order to detect faces, estimate age/gender or recognize objects. This makes usage of cloud face detection and recognition extremely hard to implement in compliance with various privacy regulations and in particular GDPR in Europe.

Our client-side approach (i.e. executing all video and image data processing in the SDK without sending data to the cloud) ensures much better privacy of user data and images.

Our SDK technology was built upon a powerful insight, that the processing capabilities of smartphone devices would increase rapidly, enabling us to create new types of personalized, anonymized and secure experiences, within the device’s browser, without the need for server side.

How scalable is the SDK?

Server-side face recognition technology must contend with the performance impact of latency, bandwidth and server side computing load.

A client-side solution does not suffer from these drawbacks. It has much more predictable performance behavior and it can easily scale to millions of users.

Client-side’s low latency allows for real-time tracking like the emotion tracking demo, since no data needs to travel from the application to the cloud and back for processing.

Why a browser HTML5 SDK instead of an app SDK in native language or c++ library?

Browser-based SDKs allow a much broader range of application and much simpler distribution to users. They only need to have access to a mobile browser instead of having to download specific native mobile apps.

Why not simply assemble a C++ library in WebAssembly to create an SDK for HTML5?

Competing machine vision technologies are focused on C++. Although several compilers are available to “translate” C++ to Javascript, their effectiveness deteriorates rapidly when dealing with something as complex as our deep neural network. Instead, we have optimized our solution module by module so as to only load code that is strictly required by the various browsers and devices to run our specific neural network.

Our approach was not easy to implement. Only a team that has been working on browser technology for years, like the MorphCast team, can develop and optimize sophisticated solutions for Javascript and browser engines.

We were able to limit our SDK to under 1MB in size and make it run fast on a variety of browsers. This accomplishment was also possible thanks to our R&D engineers and collaborations with top research universities such as CVC in Barcelona and MICC in Florence.

Deep learning vs. traditional computer vision, which is better? Which technology does MorphCast SDK use?

Traditional computer vision algorithms require a significant manual fine tuning by researchers and it only works sufficiently well when applied to common and well-studied problems such as face analysis. Deep learning approaches, on the other hand, tend to be completely data-driven, easily adaptable to different problems and suitable to tackle complex tasks.

Typically, a traditional computer vision solution is faster but with lower accuracy than a deep learning approach. This is why the MorphCast SDK uses a hybrid solution to exploit the best characteristics of the two approaches. In particular the SDK detects faces using traditional computer vision and analyzes them with deep neural networks.

MorphCast SDK also works on object recognition and it is expandable to other deep learning tasks like text analysis and generative models (GAN) maintaining the same core.

Why are the size of the Deep Neural Network model and of the Javascript code are important in a HTML5 SDK?

As soon as your web page is loaded, the SDK must be ready to react to the viewer e.g. detect a face, emotions, age, gender…) If the page load time exceeds just a few seconds, the user will think that it is not working at all (according to Google, 53% of mobile site visitors leave a page that takes longer than 3 seconds to load.)

Our SDK, at less than 800 KB in size (and potentially even less, based on the modules being used), starts in under 2 seconds in 95%+ of the cases.  

How is the MorphCast SDK different from Visage or Affectiva?

The MorphCast SDK is much “lighter” than Affectiva, approximately by a factor of 10x.

In term of technology, the MorphCast SDK uses a DNN (Deep Neural Network) approach for face analysis (i.e. emotion, age, gender and 3D head pose recognition), while Visage and Affectiva use a traditional approach for these tasks.

The MorphCast SDK also provides an Object Analysis module, to recognize, for example, household items, branded products, cars, signage, as well as a Facial Features module to recognize, for example, facial hair, jewelry, make up, hairline, glasses, etc…

What platforms does the MorphCast SDK support?

The MorphCast SDK is fully HTML5-compatible, so no app coding is necessary. Its in-browser technology does not need to upload video data to cloud servers. It runs on virtually any desktop, mobile or other browser without any user or developer intervention.How can the MorphCast SDK fluidly run deep learning models on a browser?

Typical deep learning models are large in terms of file size (tens or hundreds of MBs) and computationally intensive, so the common approach is to expose AI services through cloud APIs.

The MorphCast team, instead, has developed and tuned lightweight deep learning models that are capable to achieve better performance than other solutions, both in terms of size and execution time.

Reduced execution time is achieved by using backends like WebAssembly and WebGL, fully exploiting GPU capabilities of the client device, thereby reaching, on mobile, faster execution times than CPU-native apps.

How is MorphCast SDK different from Google Tensorflow.js? Is it a competitor?

Tensorflow.js is a library that allows to run deep learning model (in Tensorflow format) on a  browser. It is essentially a “naked” engine and, aside from some examples, it does not provide market-ready solutions.

The MorphCast SDK uses instead the WebDNN library as the engine, under the hood, to run deep learning models on the browser. WebDNN is one of the open source projects to which MorphCast team itself contributes.

How many hours a Javascript programmer needs to integrate the MorphCast SDK in a website?

Our clients report on average the integration requires 2 to 3 hours, starting from scratch. We developed the SDK with “ease of use” as our most important priority.

What kinds of things can the MorphCast SDK be used for?

Our clients are discovering many innovative and effective uses for the MorphCast SDK:

  • It enables an emotional and demographic recognition layer to be easily added to websites, enabling them to adapt to both the type of user and their mood.
  • It can be integrated into e-commerce stores and campaign landing pages, enabling fashion or style brands to change dynamically product selection and recommendations based upon the attention and reaction of the user.
  • It can be used for completely new forms of interactions, such as games that are played by tracking head position or triggered by facial expressions.
  • It can be trained to recognize any identifiable object, such as a brand logo, which can unlock more content or open up an incentive or competition entry.
  • It enables brands to serve age or demographic appropriate content to users, for example ensuring children only see content they are supposed to, whilst fully protecting the child’s identity.
  • It offers a completely new form of navigating news content, which could be used by news organizations to keep people more engaged by adapting content to the user.

One of the key benefits of being able to respond to emotional and demographic inputs from the user is the ability to deliver personalized experiences in real-time, which starts to close the online attention deficit that many sites and campaigns suffer from.

What sectors/industries can use the MorphCast SDK?

The MorphCast SDK is proving very useful to many different industry sectors and enables you to group your prospects into relevant categories to create personalized functions based on your understanding of their needs.

  • In marketing we are starting to see the very first adaptive campaigns come to life such as drinks brands that require a certain type of behavior from the user.
  • In the film, music and entertainment industries our technologies have the potential to create a completely new paradigm for artists and entertainers to engage with their fans.
  • For shopping malls, theme parks and out of home environments our SDK can be used to enrich the physical ‘live’ space in a way that fully maintains data privacy.
  • Healthcare is one of the major sectors seeing innovation, and our adaptive client-side technology ensures anonymity of the patient can be maintained.
  • Sports and sponsorship generates huge levels of emotional response from fans and is tailor made to benefit from the MorphCast SDK.

There are many other industry sectors that can use MorphCast SDK technology to segment customers and tailor the experience to match each group’s needs.