Real-Time High-Fidelity Facial Performance Capture

Share this & earn $10
DisneyResearchHub
Published at : 08 Nov 2020
422434 views

Link to publication page:
http://www.disneyresearch.com/realtimeperformancecapture

We present the first real-time high-fidelity facial capture method. The core idea is to enhance a global real-time face tracker, which provides a low-resolution face mesh, with local regressors that add in medium-scale details, such as expression wrinkles. Our main observation is that although wrinkles appear in different scales and at different locations on the face, they are locally very self-similar and their visual appearance is a direct consequence of their local shape. We therefore train local regressors from high-resolution capture data in order to predict the local geometry from local appearance at runtime. We propose an automatic way to detect and align the local patches required to train the regressors and run them efficiently in real-time. Our formulation is particularly designed to enhance the low-resolution global tracker with exactly the missing expression frequencies, avoiding superimposing spatial frequencies in the result. Our system is generic and can be applied to any real-time tracker that uses a global prior, e.g. blend-shapes. Once trained, our online capture approach can be applied to any new user without additional training, resulting in high-fidelity facial performance reconstruction with person-specific wrinkle details from a monocular video camera in real-time.
ACM Siggraph 2015Disney Research ZurichReal-time face reconstruction

Can Liverpool win the title without Virgil Van Dijk? | Souness & Carragher on VVD injury

Can Liverpool win the title without Virgil Van Dijk? | Souness & Carragher on VVD injury

The Herald 250 is a straightforward bike from another era

The Herald 250 is a straightforward bike from another era

Getting Rid of Everything + Becoming a Minimalist

Getting Rid of Everything + Becoming a Minimalist

FREE AUDIO- 'If I Quit However, It Will Last Forever'

FREE AUDIO- 'If I Quit However, It Will Last Forever'

The clothes we wear | DW Documentary

The clothes we wear | DW Documentary

Айым Айылчиева- Аяна Касымова "Ата-эне" (Жолдубай Кайыпов)

Айым Айылчиева- Аяна Касымова "Ата-эне" (Жолдубай Кайыпов)

The moment Alex Kingston learns who 13th Doctor is going to be

The moment Alex Kingston learns who 13th Doctor is going to be

The Art of Lazy Composting | How to Make High-Quality Compost the Simple Way

The Art of Lazy Composting | How to Make High-Quality Compost the Simple Way

Seek - Totul In Doi

Seek - Totul In Doi

Minimize Card Game - A Detailed Explanation !

Minimize Card Game - A Detailed Explanation !

بعد أعوام من الآن ستتمنى أنك بدأت اليوم | محمد الملحم | TEDxRiyadh

بعد أعوام من الآن ستتمنى أنك بدأت اليوم | محمد الملحم | TEDxRiyadh

Trying Wild Celebrity Breakfast Foods | People Vs. Food

Trying Wild Celebrity Breakfast Foods | People Vs. Food

5 Risk Factors for Heart Disease | Cedars-Sinai

5 Risk Factors for Heart Disease | Cedars-Sinai

I got to drive the world's LARGEST motorcycle

I got to drive the world's LARGEST motorcycle

케이팝_0.5

케이팝_0.5

Data Science: Reality vs Expectations ($100k+ Starting Salary 2018)

Data Science: Reality vs Expectations ($100k+ Starting Salary 2018)

Emerson, Lake & Palmer: Take A Pebble (1970)

Emerson, Lake & Palmer: Take A Pebble (1970)

THERE IS / THERE ARE - САМОЕ ПРОСТОЕ ОБЪЯСНЕНИЕ С ТЕСТОМ │ English Spot - разговорный английский

THERE IS / THERE ARE - САМОЕ ПРОСТОЕ ОБЪЯСНЕНИЕ С ТЕСТОМ │ English Spot - разговорный английский

Weekend Special: Christmas Musical—Light In Our Dwellings | Premiering Fri, 18 Dec, 8pm SGT

Weekend Special: Christmas Musical—Light In Our Dwellings | Premiering Fri, 18 Dec, 8pm SGT

Transport Services - (Lightweight Transportation Simulation Game)

Transport Services - (Lightweight Transportation Simulation Game)

Wat a wonderfully world

Wat a wonderfully world

Giving Away A Free Laptop to College Students

Giving Away A Free Laptop to College Students

Ingathering 2020

Ingathering 2020

2009-2013 Silverado Spectre Performance Polished Cold Air Intake 5.3L Review & Install

2009-2013 Silverado Spectre Performance Polished Cold Air Intake 5.3L Review & Install

Easy Piano Tutorial: Lightly Row with free printable PDF sheet music

Easy Piano Tutorial: Lightly Row with free printable PDF sheet music

Method Man, Redman - Da Rockwilder (Official Video)

Method Man, Redman - Da Rockwilder (Official Video)

Tension and Compression Forces in Buildings.

Tension and Compression Forces in Buildings.

All Is Found from "Frozen 2" (Kacey Musgraves) | The Hound + The Fox

All Is Found from "Frozen 2" (Kacey Musgraves) | The Hound + The Fox

Is Dude Perfect Fake?

Is Dude Perfect Fake?

CGI Animated Short Film HD "Dead Friends " by Changsik Lee | CGMeetup

CGI Animated Short Film HD "Dead Friends " by Changsik Lee | CGMeetup

What is Counterpoint?

What is Counterpoint?

Kim Bum Soo - Appear // Sub Español

Kim Bum Soo - Appear // Sub Español

[MV] 10cm _ Phonecert (폰서트)

[MV] 10cm _ Phonecert (폰서트)

I am the victim of a child assault, how dare they boo me.

I am the victim of a child assault, how dare they boo me.

Harry Potter and the Prisoner of Azkaban

Harry Potter and the Prisoner of Azkaban

xQc Reacts To Daily Dose Of Internet!

xQc Reacts To Daily Dose Of Internet!

Sam and Dean II I am proud of us (15x20)

Sam and Dean II I am proud of us (15x20)

Monster gaming//garena free fire//me majbur hu 50% discotion and loos

Monster gaming//garena free fire//me majbur hu 50% discotion and loos

International Relations: Diplomacy |History |Definition |Importance |Functions & Types of Diplomacy

International Relations: Diplomacy |History |Definition |Importance |Functions & Types of Diplomacy

PREFERRED SHARES BUY BACK

PREFERRED SHARES BUY BACK

9 year old girl surprises singing "Rise up" in America's Got Talent.

9 year old girl surprises singing "Rise up" in America's Got Talent.

Doug Ford’s COVID protocols are clear as mud | 22 Minutes

Doug Ford’s COVID protocols are clear as mud | 22 Minutes

The Development of the K computer

The Development of the K computer

Looking for Gold in Clear Blue Water👑Dentex the Queen of Gold |Spearfishing Life 🇬🇷 [4K]✅

Looking for Gold in Clear Blue Water👑Dentex the Queen of Gold |Spearfishing Life 🇬🇷 [4K]✅

fishbone a selection

fishbone a selection

Curiosity | 10 Days of Genius | National Geographic

Curiosity | 10 Days of Genius | National Geographic

Married at First Sight l When marrying a stranger works out perfectly

Married at First Sight l When marrying a stranger works out perfectly

9 Regions of Abdomen made simple

9 Regions of Abdomen made simple

Acoustic Jam @津とまんなかjazz festival "especially true"

Acoustic Jam @津とまんなかjazz festival "especially true"