This project is a Web Application showcasing the Facial Landmark Detection using Tensorflow.js. We developed the Lipsync game for the user to sing along (using only the mouth shape input) and use Machine Learning to detect the mouth shape and calculate the score based on the accuracy of the player’s mouth shape.
“Lipsync” has been showed on the keynote stage and exhibited at Tensorflow World 2019 in Santa Clara, CA.
Track AI is a project we use machine learning models for Visual pathologies detection to examine and identify children with visual disorders as early as possible.
The visual system of humans is not fully developed at birth and gradually improves during the first few years of life. World Health Organization estimates there are about 19 million children in the world suffering from visual impairment. However, 70–80% of them could prevent or treat it, if they have been detected with visual disorders in childhood.
Using an eye tracker with the patient, we could know which point of the screen that the patient is looking at, but analyzing gaze data of the patient is quite complicated, that is why we choose machine learning to help ophthalmologists to diagnose visual pathologies. We could collect data by performing various tests on the patients, and the ophthalmologist supervises this operation.
Thanks to DIVE, a visual examination device, we have lots of gaze data from children. We used that gaze data to feed into the ML model, and some tests also made use of the ASA (Accelerated Stochastic Approximation) algorithm to provide meaningful constants before using in the ML model. Outputs of the model are the probability of abnormalities for each eye and each pathology.
Imagine we could take the Japanese Hanami festival to Bangkok!
Introducing “The Cherry Blossom Forest”, our latest interactive flower tunnel that will not only give you a magical experience, but also respond to a flick of your touch. Walk among the stream of petals, take photos with the elegant flowers, and immerse yourself in the tunnel filled with cherry blossoms.
Our installation was exhibited at the walkway between Central Chidlom and Central Embassy, Bangkok in November 2019.
Pixelopolis is a project we designed and developed for Google to promote the launch of Google Pixel 4 in Japan. Our goal is to create a playful experience for customers by focusing on the power of AI/ML using the Neural Core on Google Pixel 4.
Using only the front camera on Pixel 4, we are able to do the lane-keeping, localization and object detection right on the device in real-time. Not only that, the Pixel 4 also control the motors and other electronic components via USB-C, so the car can stop when it detects other cars or turns at a right interaction when it needs to. Since user needs to select a destination for a car with a Station phone, we also do a real-time video streaming from the car directly to the Station phone, so the user can see what’s going on.
Seeing is believing. Please visit our latest and greatest installation at Omote-Sando in Tokyo today!
“Morphing Clay” is an AI interactive Installation based on the design of the ancient chinese pottery that we developed with Google using Tensorflow and Unity. Nine body poses come nine different pottery shapes. If the players show their hands up in the air and wider their arms, our pottery would become taller. Moreover, if they move their body to the left and right, our pottery would lean following them.
More interestingly, the textures of our pottery also influenced by what the players wear. As we use AI to do the style transfer by integrating the player’s shirt patterns with the old chinese designs, this brings up the special pottery textures. Wrap these ancient pottery with these unique textures and they become an exclusive pottery.
We had installed our Morphing Clay installation at the WAIC 2019 and GDD China 2019 in Shanghai, China.
PLAY ME is an interactive installation simply explain how the core feature, Balance Sweep, works. Since the feature itself is quite complicated to understand, we design a way to explain it in a simple and magical way.
To begin with, in every month user generates income on ME MOVE account. With Balance Sweep, the feature will automatically transfer the remaining amount to ME SAVE account and allow user to receive an exceptional rate of interest. The user will be amazed with the amount saved on their account!
PLAY ME is designed based on user’s behavior. There are 3 actions to play;
First, generate your own income with Earning box by just spinning the handle. The more you spin, the more you earn! Second, save more money with Balance sweep feature. Customize your savings amount by interacting with the touch screen in front of you. Third, see your return. By opening the safebox, you will see how much more you can save with ME!
Identifying your spending habits with TOUCH ME!
TOUCH ME is a quick & easy to play touch screen game. Each round you will have to choose between two spending option within 3 seconds. The result of the game will reveal the type of spending habit you are! Amazed by the results, many players love to share the results on their social media. This emphasizes that ME digital banking is matched with their lifestyles.
Together with Pentagram, we developed a centralized app for Kathleen Grimm School for Leadership and Sustainability, a “Net Zero school”, aiming to make as much energy as it uses everyday. Our app displays information related to Net Zero such as the amount of energy the school generated and uses each day, the location of solar panels, electric bikes and various useful tips on how to conserve energy.
Information from sensors installed throughout the school is queried by the app installed on each floor. The app then display that information in an easy to understand yet stylish way. Although our app is designed to capture children’s interest, we also find that adults are also attracted to them too.
Collaborated with Pentagram, we developed an app for the sales gallery of the 53w53 condominium. There are two versions of the app, the PC version and the iPad version. The PC version is installed on an on-site PC and operates on a touch screen and a 4K video wall, allowing great details to be explored on the grandiose screen. The iPad version, on the other hand, lets the salesperson approach more customers at the sales gallery, while allowing full information to be viewed with great comfort. We optimized the experience so that both versions of the application runs smoothly and able to display similar information despite having limited computational power.
The app is operated by salesperson at the residence gallery. The app searches for rooms that match customer criteria such as price, number of bedrooms and room size. The floorplan of the apartment rooms can be explored in great detail using touch gestures, customers can also compare any two floor plans or view photos taken from each floor in panoramic view. The app also display details such as condominium amenities, landmarks around the place and the designers’ other works.
Customers can mark any rooms as interested and the app will send details to their email, so that they can explore the rooms at their leisure.
Touchscreen and video wall
iPad
Together with Pentagram, we enliven the stage for the opening day of The Exponential Center of the Computer History Museum in Mountain View, California. Emphasized by the elements of pixels and golden ratio, we computationally process videos into pixelated art and render them on the screen in real time, which are then duplicated in an infinite golden rectangle.
