Work

StageLab: The first-ever interactive AR experience for live concerts

bit.studio, partnering with Google and BUCK, collaborated with ModernSky and New Pants to deliver an innovative stage performance that enhances live concert experience.

At Strawberry Music Festival 2020, we created an AI/ML powered AR experience to enliven the concert like never before. The system was applied to New Pants’ live stage where certain actions of the artists and audience activated live AR effects to create a joyful interactive concert experience.

Powered by AI/ML, the motion graphics are automatically generated, scale, rotate, and position over the concert camera feed in response to specific objects, poses, or gestures of the artists and audience. Our system incorporated object detection and orientation, face detection, facial landmark detection, pose detection, gesture detection, and crowd plane detection. Everything you see on the live screen is detected and rendered in real-time – no post-production required!

Tech: Augmented Reality (AR), AI/ML – Crowd Plane Orientation, Guitar Orientation, Microphone Orientation

Experience the never-been-done-before interactive experience on the YouTube Live videos with FanReact!

FanReact is an interactive live performance visual effects powered by live YouTube comments that we have designed and developed for YouTube FanFest 2020. The audience can show their love to their favorite YouTube artists by simply commenting “♥️” to the live comments and our software will automatically create a visual effect in the live video using their profile pics in real-time!

This is the first time ever that any audience can be a part of a live video alongside the content creators on YouTube. Get more out of a live concert with real-time interaction between audiences and creators. Surprised your audiences with our interactive effect across the show!

Contact us at contact@bit.studio if you want this experience to be integrated into your live performances!

Check out some of the impressions on social media!

Do you want to be part of the dance movement with Matt Steffanina?

Try syncing your dance, share it out, and challenge your friends here!

bit.studio as part of Youtube and Google team, has created a new dance experience for Youtube FanFest 2020. We developed a software with a Machine Learning model that allows users to enjoy their dance movement and get their own dance video with Matt Staffanina, the most followed dance creators on YouTube. Users can download their dance syncing videos with Matt and share it out.

What you need to do is very simple. You learn the full dance in the tutorial video, then try and keep up your move with Matt. When ready, record your dance movement. And our ML model will detect your movement and segment your body out. Then, feature the moves on your dance video!

Scalability is very important in this project since we need to support millions of users concurrently. We decided to run our ML model on Google Kubernetes Engine with dedicated graphic cards. This way we can generate the videos with just a short amount of time.

We had so much fun creating such an enjoyable experience for YouTube FanFest 2020. Now it’s your turn! Let’s dance! yt.be/ytffdance

If you are interested in this experience and would like to know more, feel free to contact us at contact@bit.studio.

Tech: AI/ML – Body Segmentation, Web application, Online experience

Mitrtown Office Tower is a Grade A office building, located in the CBD of Bangkok. This Intelligent office tower was designed under the concept of “Smart & Friendly”. Our team, bit.studio, took a role to give users creative and fresh daily content through various screens in the reception area, elevators’ hall and outdoor facade.

To create an experience that matches our users, we have to know who they are, what are their shared interests which can be provided by the given screens, their attention, and their behavior or activities that can happen in the lobby. From our site survey and observation, we found that our target audience is office workers working in the building. Their attention corresponds to their walking path starting from the lobby LED, which catches their attention the most yet takes their time the least, to the reception video wall, and the elevator LED, which users will spend their time with it the most while they are waiting for the elevator. Time is also one of major factors that affect users’ interests which differ from each period of time during the day.

Other than our target audience, we also looked into people’s behavior with similar types of media, such as screens which are mostly seen in mass transit systems. From our observation, people tend to ignore public screens. Not that they intuitively ignore the screens but they develop a habit of ignoring commercials on the screen. With this perspective in our mind, we aimed to create a new user habit which makes users want to take a glance at the screens everyday, to make the most of all the screens in the lobby. To create such a habit, we chose to deliver users’ interests at the right time. The content should be expectable yet not boring in order to keep users’ attention with their interested information and pleasant ever-changing visuals.

Design Development
All these aspects considered, we came up with a ‘theme and trigger’ system, using a timeline to select which content to be displayed. Every screen displays the same theme in a specific period of time. We developed 3 computer generated themes that match office workers’ interests during each period of the day.

The main theme ‘Capsule’ was designed to retain user attention with a captivating capsule system design that developed from trails of a moving circle, and inform basic information: date, time, and outdoor weather. At the same time, customized inspirational quotes, keywords, and related symbols are also displayed to plant good thoughts and spread positive energy to people working at or visiting the Mitrtown Office Tower.

The second theme ‘Vector Field’ illustrates the local Air Quality Index through an animated vector field that makes each screen look like a floating fabric. The theme is triggered 3 times a day: 8-9 A.M when most office workers arrive at their offices, 12-1 P.M. when they go outside for lunch, and 6-7 P.M. when they leave the office, in order to get them prepared for a bad air day or help them feel safe to breathe in the clean air. Colors matching with the US AQI scales are embraced as the background of the animated vector field to make users aware of the air quality level with just a single glance.

The third theme ‘Isometric Cube’ focuses on stock index, informationally and visually. Data from each market is visualized through kinetic cubes, a landscape of moving cubes, adding movements to the architecture. The cubes are manipulated to form into digits and graphs to inform stock index and growth of the market.

Technical challenges
As mentioned above, the project visualizes data on 5 different screens, each connected with a computer to generate graphics in real time. There is one dedicated server who collects data from various sources. It keeps the schedule and triggers all the frontends when the theme has to be changed. The server provides data to all the frontends which turn it into graphics through algorithms. The entire visualization is programmed and generated using Unreal Engine 4. The server is written in Python.

To achieve the design concept, we set up 3 different virtual spaces. Each space has its own environment, lighting and data visualization system. There are 5 3D-cameras located in the space, capturing real time scenes.

The ‘Capsule System’ theme turns data into fonts and 3d paths, then algorithmically generates spheres which leaves trails as they travel along the paths.

Technically, the challenge of the ‘Vector Field’ theme is to create an animated vector field. A custom shader is made for this theme in order to turn solid cylindrical geometries into a fabric-like plane. This ‘fabric’ waves according to the local wind speed.

We developed the kinetic cubes as a key for the ‘Isometric Cube’ theme. The realistic light and shadow of these cubes is rendered with realtime ray tracing. This information is passed to a custom shader which determines the pillar movements.

This project is one of a great combination of design development, which matches perfectly with user’s behavior, and technical challenges, which generates a real time visually and informationally graphics based on the design requirements.

Thanks to the clients for taking the journey of developing ‘Mitrtown Data Sculpture’ with us. Lastly, one thing that we believe is that we can create a good impact on users’ life by using creative technology in a creative new way, and this will change their life since then.

If you are looking for a creative yet functional installation to be integrated in your space, please feel free to contact us at contact@bit.studio.

Tech: Data Visualization, Generative Art