Emotional Facial Recognition AI HTML5 SDK FAQ

What is MorphCast AI HTML5 SDK?

MorphCast AI HTML5 SDK is a native small and powerful JavaScript engine (less than 1MB), based on Deep Neural Networks AI, able to analyse facial expressions and features in any web page or app. MorphCast’s facial expressions analysis technology allows you to understand who the user is, detecting and analysing 130+ facial features to drive and measure viewer’s engagement.

It works directly in the web-browser of mobile and desktop and in a webview inside mobile App. It fires events at an average rate of 10 times per second on mobile, and even up to 30 per second on desktop.

Data output is ready-to-use, already filtered for your convenience (parameters can also be changed in order to have a smoother output for more convenient use in your code). You can store all data produced in local memory, in local storage or properly send it to your server.

This SDK was developed with you in mind, to have a really quick integration into your application.

Try MorphCast AI Demo to discover how AI HTML5 SDK works, and which features it is able to detect and analyze.

 

How is MorphCast AI HTML5 SDK different from alternatives?

Our AI technology is fully GDPR compliant, so privacy is respected 100%.

No cloud computing costs allow the use in all fields where the cost of the same features would be prohibitive: the price is up to 1000 times cheaper than other providers.

The size of MorphCast AI HTML5 SDK is under 1MB: it is much more “lighter” than alternatives, approximately by a factor of 12x, and it is very fast.

In terms of technology, the MorphCast AI HTML5 SDK uses a CNN (Deep Convolutional Neural Network) approach for face analysis (i.e. emotion, age, gender and 3D head pose recognition).

MorphCast AI HTML5 SDK ensures full flexibility, thanks to its modular architecture, and full scalability, thanks to its client side approach that can easily scale to millions of users.

Finally it is an in browser technology (javascript), working across desktop, mobile and digital display and it’s super easy to integrate, with a few clicks you’ll embed the snippet and be ready to go. 

Can MorphCast AI HTML5 SDK identify people? How do you handle the privacy of user data? Does MorphCast AI HTML5 SDK store or send sensitive data to others?

MorpCast takes your privacy and data-safety very seriously. Our AI facial analysis technology doesn’t record or archive any personal data: MorphCast’s AI uses the device's camera as a sensor, not as a video image recorder. It has been specifically designed and developed to NOT identify people. For this reason, MorphCast’s AI technology can be considered totally safe and risk-free.

In addition, privacy is guaranteed by the fact that MorphCast accesses personal data (face image) only if granted by the user; only for the time strictly necessary to process the features (about 100 milliseconds), only when the application is active and only for the stated purposes. Such images are never transmitted, stored or shared with others. For more background on this, we strongly recommend reading the GDPR documentation.

How can I get MorphCast AI HTML5 SDK full features licence?

You can get an AI HTML5 SDK full feature licence simply filling out this form. Then just follow our instructions and you’ll have the SDK live on your HTML page in just 3 minutes.

It’s super easy to integrate, with few clicks you’ll embed the snippet and be ready to go.

 

How much does MorphCast AI HTML5 SDK cost?

The price of MorphCast AI HTML5 SDK is calculated based on the number of views and the average uptime of the AI HTML5 SDK for each view. The minimum charge is one minute per view.

To have an idea of the price you would pay based on your expected monthly volumes, please try our price calculator.

What is able to recognise MorphCast AI HTML5 SDK?

MorphCast AI HTML5 SDK is a modular library containing the following modules/outputs:

  • Faces: Presence of faces from 0 to 6
  • Pose on the tree axes: pitch, roll, yaw
  • Age, Gender
  • Emotions: Angry, Disgust, Fear, Happy, Sad, Surprise, Neutral (according to the P. Ekman discrete model)
  • 37 face features: Arched Eyebrows, Attractive, Bags Under Eyes, Bald, Bangs, Beard 5 O'Clock Shadow, Big Lips, Big Nose, Black Hair, Blond Hair, Brown Hair, Chubby, Double Chin, Earrings, Eyebrows Bushy, Eyeglasses, Goatee, Gray Hair, Hat, Heavy Makeup, High Cheekbones, Lipstick, Mouth Slightly Open, Mustache, Narrow Eyes, Necklace, Necktie, No Beard, Oval Face, Pale Skin, Pointy Nose, Receding, Hairline, Rosy Cheeks, Sideburns, Straight Hair, Wavy Hair.
  • Arousal and Valence according to the dimensional model of Russell.
  • 38 affects (circumplex model of affect, Paltoglou & Thelwall scientific paper reference): afraid, amused, angry, annoyed, anxious, apathetic, astonished, bored, calm, conceited, contemplative, content, convinced, delighted, depressed, determinated, disappointed, discontented, distressed, embarrassed, enraged, excited, fell well, frustrated, happy, hopeful, impressed, melancholic, peaceful, pensive, pleased, relaxed, sad, satisfied, sleepy, tired, uncomfortable, worried.
  • 98 affect (scientific reference Russell, J. A., Lewicka, M., & Niit, T.Klaus R. Scherer): Adventurous, Afraid, Alarmed, Ambitious, Amorous, Amuseds,Angry Annoyed, Anxious, Apathetic, Aroused, Ashamed, Astonished, At Ease, Attentive, Bellicose, Bitter, Bored, Calm, Compassionate, Conceited, Confident, Conscientious, Contemplative, Contemptuous, Content, Convinced, Courageous, Defiant, Dejected, Delighted, Depressed, Desperate, Despondent, Determined, Disappointed, Discontented, Disgusted, Dissatisfied, Distressed, Distrustful, Doubtful, Droopy, Embarrassed, Enraged, Enthusiastic, Envious, Excited, Expectant, Feel Guilt, Feel Well, Feeling Superior, Friendly, Frustrated, Glad, Gloomy, Happy, Hateful, Hesitant, Hopeful, Hostile, Impatient, Impressed, Indignant, Insulted, Interested, Jealous, Joyous, Languid, Light Hearted, Loathing, Longing Lusting, Melancholic, Miserable, Passionate, Peaceful, Pensive, Pleased, Polite, Relaxed, Reverent, Sad, Satisfied, Selfconfident, Serene, Serious, Sleepy, Solemn, Startled, Suspicious, Taken Aback, Tense, Tired, Triumphant, Uncomfortable, Wavering, Worried.
  • One of the four quadrants in the circumplex model of affect: "High Control", "Obstructive", "Low Control", "Conductive", or "Neutral" (scientific reference Russell, J. A., Lewicka, M., & Niit, T.Klaus R. Scherer)

MorphCast AI HTML5 SDK can also be trained to recognize objects, for example, household items, branded products, cars, signage etc. 

Do people open the camera?

Our solution, based on our metrics and our clients’ feedback, achieves camera open rates ranging from 63% to 88%, depending on the context and the content/creative.

Privacy is guaranteed by the fact that the AI HTML5 SDK accesses personal data (face image) only for the time strictly necessary to process the result, only if granted by the user, only when the application is active and only for the stated purposes. Such images are never transmitted, stored or shared with others. For more background on this, read the GDPR documentation.

Why a client-side approach?

One of the primary reasons why we decided to take a client-side approach is user privacy.

Cloud computing services, including cloud-based machine vision APIs require that video frames and images are sent to a server to be analyzed in order to detect faces, estimate age/gender or recognize objects. This makes usage of cloud face detection and recognition extremely hard to implement in compliance with various privacy regulations and in particular GDPR in Europe.

Our client-side approach (i.e. executing all video and image data processing in the AI HTML5 SDK without sending data to the cloud) ensures much better privacy of user data and images.

Our AI HTML5 SDK technology was built upon a powerful insight, that the processing capabilities of smartphone devices would increase rapidly, enabling us to create new types of personalized, anonymized and secure experiences, within the device’s browser, without the need for a server.

 

You say that MorphCast AI HTML5 SDK is less than 1MB, but what about the accuracy?

The accuracy and performance of MorphCast AI HTML5 SDK are continuously updated. One of the first versions of MorphCast AI HTML5 SDK was tested by independent research: “Dupré, D., Krumhuber, E., Küster, D., & McKeown, G. J. (2019, September 26). Emotion recognition in humans and machines using posed and spontaneous facial expression.” Link: https://psyarxiv.com/kzhds/. In this independent research you can find that MorphCast's accuracy is quite comparable to that of competitors who use dedicated servers in the cloud to run large networks. In some cases higher.

How scalable is AI HTML5 SDK?

Server-side face analysis technology must contend with the performance impact of latency, bandwidth and server side computing load.

A client-side solution does not suffer from these drawbacks. It has much more predictable performance behavior and it can easily scale to millions of users.

Client-side’s low latency allows for real-time tracking like the emotion tracking demo, since no data needs to travel from the application to the cloud and back for processing.

Why a browser HTML5 SDK instead of an app SDK in native language or c++ library?

Browser-based SDKs allow a much broader range of applications and much simpler distribution to users. They only need to have access to a mobile browser instead of having to download specific native mobile apps.

Why not simply assemble a C++ library in WebAssembly to create an SDK for HTML5?

Competing machine vision technologies are focused on C++. Although several compilers are available to “translate” C++ to Javascript, their effectiveness deteriorates rapidly when dealing with something as complex as our deep neural network. Instead, we have optimized our solution module by module so as to only load code that is strictly required by the various browsers and devices to run our specific neural network.

Our approach was not easy to implement. Only a team that has been working on browser technology for years, like the MorphCast team, can develop and optimize sophisticated solutions for Javascript and browser engines.

We were able to limit our AI HTML5 SDK to under 1MB in size and make it run fast on a variety of devices. This accomplishment was also possible thanks to our R&D engineers and collaborations with top research universities centers such as CVC in Barcelona and MICC in Florence.

Deep learning vs. traditional computer vision, which is better? Which technology does MorphCast AI HTML5 SDK use?

Traditional computer vision algorithms require a significant manual fine tuning by researchers and it only works sufficiently well when applied to common and well-studied problems such as face analysis. Deep learning approaches, on the other hand, tend to be completely data-driven, easily adaptable to different problems and suitable to tackle complex tasks.

Typically, a traditional computer vision solution is faster but with lower accuracy than a deep learning approach. This is why the MorphCast AI HTML5 SDK uses a hybrid solution to exploit the best characteristics of the two approaches. In particular the AI HTML5 SDK detects faces using traditional computer vision and analyzes them with deep neural networks.

Why are the size of the Deep Neural Network model and of the Javascript code, are important in a HTML5 SDK?

As soon as your web page is loaded, the AI HTML5 SDK must be ready to react to the viewer (i.e. detects a face, emotions, age, gender…). If the page load time exceeds just a few seconds, the user will think that it is not working at all (according to Google, 53% of mobile site visitors leave a page that takes longer than 3 seconds to load).

Our AI HTML5 SDK, at less than 1 MB in size (and potentially even less, based on the modules being used), starts in under 2 seconds in 95%+ of the cases.  

 

What platforms does the MorphCast AI HTML5 SDK support? Does it need any user or developer intervention to run?

The MorphCast AI HTML5 SDK is fully HTML5-compatible, so no app coding is necessary. Its in-browser technology does not need to upload video data to cloud servers. It runs on virtually any desktop, mobile or other browser without any user or developer intervention. 

How can the MorphCast AI HTML5 SDK fluidly run deep learning models on a browser?

Typical deep learning models are large in terms of file size (tens or hundreds of MBs) and computationally intensive, so the common approach is to expose AI services through cloud APIs.

The MorphCast team, instead, has developed and tuned lightweight deep learning models that are capable of achieving better performance than other solutions, both in terms of size and execution time.

Reduced execution time is achieved by using backends like WebAssembly and WebGL, fully exploiting GPU capabilities of the client device, thereby reaching, on mobile, faster execution times than CPU-native apps.

How is MorphCast AI HTML5 SDK different from Google Tensorflow.js? Is it a competitor?

Tensorflow.js is a library that allows you to run deep learning models (in Tensorflow format) on a browser. It is essentially a “naked” engine and, aside from some examples, it does not provide market-ready solutions.

MorphCast AI HTML5 SDK uses instead the WebDNN library as the engine, under the hood, to run deep learning models on the browser. WebDNN is one of the open source projects to which the MorphCast team itself contributes.

How many hours a Javascript programmer needs to integrate the MorphCast AI HTML5 SDK in a website?

Our clients report, on average, that the integration requires 3 to 5 minutes, starting from scratch. We developed the AI HTML5 SDK with “ease of use” as our most important priority. For getting a licence key, please fill the form in this page.

What kind of things can  MorphCast AI HTML5 SDK be used for?

Our clients are discovering many innovative and effective uses for MorphCast AI HTML5 SDK:

  • It empowers you to drive and increase customer engagement in real-time by creating personalized, interactive experiences led by user's facial expressions.
  • It allows you to evaluate the effectiveness of your contents and messages, by measuring the level of attention, arousal and emotional response of your users, and use these data to better customize future communication in terms of contents, tone of voice and look & feel.
  • It allows you to deliver the right content to the right person through analysis of their likely demographics data and unique features.

Does my device/app need Internet connection to run MorphCast AI HTML5 SDK? 

MorphCast AI HTML5 SDK needs an Internet connection to be automatically downloaded and run by a regular web page. It works directly on the user’s device/browser or app (client-side approach) without remote server and API processing. Sensible data is processed in real time and not stored or sent anyway.

What sectors/industries can use the MorphCast AI HTML5 SDK?

MorphCast AI HTML5 SDK is proving to be very useful to many different industry sectors and enables you to group your prospects into relevant categories to create personalized strategies based on your understanding of their needs.

  • In digital ADV, to effectively capture attention, increase customer emotional engagement and select real people from BOTS 
  • In digital learning, to verify attendance, mood and attention of student, and improve accuracy of remote testing and student access
  • In RTC applications, to monitor the attention level, emotional state and presence of attendees during videoconference and webinar
  • In Human Resources, to scientifically verify the emotional state and mood of candidates during interviews or in playback, highlighting e tagging peaks
  • In retail and OOH applications, to personalise in-store communication in real-time and automatically gather in-depth data of customers
  • In e-Commerce, to personalise online offering and communication in real-time and automatically gather in-depth data of new / not logged users
  • In Apps and community, to gather in-depth data of users to enrich and fix the database and engage users through highly personalized experiences
  • In market research, to analyze customers response to products / services and gather information to better allocate the offering

There are many other industry sectors that can use MorphCast AI HTML5 SDK technology to segment customers and tailor the experience to match each group’s needs.