PAUL LACROIX 

Director / Technical director / Developer

Interactive content design

Sensing system development

Sound design

Floor lighting system development

This immersive experience was installed at the entrance of BANDAI SPIRITS headquarters in Tokyo. The setup consists of two 6.5m-long LED screens on both sides of a corridor.

Depending on the season, different contents are shown. Optical sensors allow these real-time interactive contents to respond to people's movements.

Consultant / Software Engineer

This project focused on developing a specialized software solution for reconstructing skull meshes in the medical field. Besides the physical limitations of the scanner (only a limited area of the skull can be captured in each scan), the software aligned and merged the multiple partial scans. This proved challenging at times, particularly because of the patient's head movements during the scans.

Artist

'Path of Noise (r, theta, phi)' received the Festival Platform Award Geo-Cosmos category at the Japan Media Arts Festival.
In this artwork, particles are moving along the surface of a sphere and are tracing curves along their passage. The movement of the particles is a symbolization of the randomness of one's life and its chaotic path.
The content is generative, in other words, different patterns and colors appear for each iteration.

A quite challenging aspect of this project was using a sphere-shaped screen (6-meter-diameter). Because there is no border, there is neither a beginning nor an end. The particles can move in all directions and loop around the sphere.

Although it is generative content, I also wanted to tell a story. The artwork is composed of several parts in which I tried to put different moods. Recent years have been difficult for many people around the world. I wanted to express some hope and bring some comfort into the heart of the viewers. That's the reason for using cherry blossoms.

Engineer

Though the system was initially conceived in order to play multiple video contents synchronized over multiple computers, a very special requirement for this installation was on the contrary the videos to be played randomly and avoid any impression of looping.

Technical Director

Technical direction

Design of the content production workflow and the templates

Calibration

This project proved to be quite challenging, primarily due to the subtle contours shaping the car's design. Achieving perfect alignment of the projection with the car was crucial to ensure visual consistency and coherence.

In addition to overseeing the technical aspects of the projection mapping, I was also in charge of establishing a workflow for the content team to develop their materials, as well as handling the calibration of the projection.

Speaker

A Story About Face And Body Projection Mapping

During this speech, I talk about the experiences and inspirations that led me to my work on face projection mapping and body projection mapping.

System developper/Calibrated system expert

Development of an application capable of synthesizing a 12K wide camera equivalent by combining three 4K cameras. A precision-focused calibration system, tailored for sports scenarios, ensures impeccable visual outcomes as it deftly stitches images side-by-side, presenting a seamless and visually correct result. The system operates at a swift 60 frames per second, delivering high-resolution panoramic views perfectly suited for competitive sports coverage and other live events.

Expanding its horizons beyond sports, this technology finds adaptable utility in the realms of live stage performances and outdoor landscapes, where it confers remarkable visual clarity and detail.

Artist

At the occasion of the moving of their headquarters to Toyosu, the company NGC commissioned me for a variation of one of my content.

Displayed on 4 curved screens, the content evolves and changes perpetually. Colors and shapes, as well as camera work are generated randomly in real-time.

Creative Technologist

Technical direction

Creation of a system (hardware & software) able to smoothly track the performer body and accurately project video animations fitting the body shape

After several years experimenting on body projection mapping, this collaboration between Daisuke Hashimoto, Yae Doi and me represents a new step in the field of body projection mapping. The video is starring the Japanese artist Mari Katayama.

Besides the cutting-edge technical aspect, we tried to create a peculiar visual, strongly inspired by the Japanese painter Ito Jakuchu. We also question the relation between identity and technology, as well as the perception of beauty and taboo within our societies.

Technical director / Software developper

Software development

Developped as a front end demo for the AI interface Watson, this hologram interacts and provides information to customers visiting the head quarters of IBM Japan.

Technical Director / Technology Developper

Technical direction

Customization of the markerless face projection system for live conditions

Eurovision Song Contest is a huge broadcasted live event followed by 200 million viewers. Part of the French delegation, I was in charge of the Face Projection Mapping. The system had to be installed on stage in only 40 seconds, so all calibrations and tests had to be completed before. I customized and optimized the system in order to increase its robustness for such a live event. The video is a little bit dark as the lighting technician forgot to switch on the backlights during the face projection, but the face tracking and projection mapping itself went well.

Director / Technology developper

Direction, content creation

System development

Before this project, most of my facial projection mapping videos were shot in studio or at least in a location with easy access to electricity and light. The Lady in the Wood is an experiment aiming to evaluate the feasibility and robustness of using markerless face detection system in an uncontrolled environment. The video was shot in a small forest near Mont Fuji.

Technical Director

Design of the technical solution and installation

30 screens (5x3x2) for a total resolution of 19200x3240 pixels

Engineer / Consultant

Consultant for high-speed tracking system and its integration

Development of a customized library for tracking on the wall

Software Engineer

Development of two interactive contents

Electronic interface for synchronizing the content with nearby elevator

Technical Director / System Developper

Development of a Mobile Facial Projection Mapping System (hardware and software)

Software Developer / Interaction Designer

Home-made position sensor

Development of two interactive contents

Director / Face Projection Mapping System Developer

Direction of the content creation

Markerless tracking customized hardware development

Developpement of a non-rigid markerless tracking and projection mapping software

Software Developer / Interaction Designer

Technical Director / 3D Tracking Engineering

Use of the high-speed sensor/projector of Panasonic

Development of a custom 3D tracking approach for the upper body in live stage conditions

Integration into Unity3D

Use of my original synchronized media player for the background projectors

Technical Director / Facial Projection Mapping Technology

Original motion capture system customized for facial projection mapping (hardware and software)

Upgrade of the facial projection mapping software Live Mapper

More technical details on my blog http://paul-lacroix.tumblr.com

Technical Director / Facial Projection Mapping Technology

Design of animated texture contents

Development of an original motion capture system customized for facial projection mapping (hardware and software)

Upgrade of the facial projection mapping software Live Mapper

Upgrade of the real-time reflective effect (mirror)

Creation of a new deep water effect

More technical details on my blog http://paul-lacroix.tumblr.com

Development and integration of an original system with a convenient workflow for content creators

Real-time full body motion capture and rendering, support of Lip sync

Support of multiple characters and props

Consultant for Body Projection Mapping

Development of interactive contents demonstrating the capabilities of the new Panasonic's High-Speed Projection Mapping system

Customization of the system for business suits

System upgrade (multi-layer, auto-skinning, support of very high-resolution textures...)

Improvement of the calibration system

System adjustments

Technical Director

Creation of a full body tracking and projection mapping system

Calibration system

Body tracking

Video texture projection

Technical Director / System Architect

New features of the Face Projection system:

Dual face projection mapping

Real-time relighting

Installation of original media server synchronizating video on multiple computers

27 screens (9x3) for a total resolution of 11520x2160 pixels (3x4K)

Adjustment of the system to handle multiple audio tracks

Technical Director / System Creator

Design & Development of the Real-time Face Tracking and Projection Mapping System:

Calibration system (projector and OptiTrack devices)

Original face tracking algorithm

Video texture projection

Software Developper / Interaction Designer

Real-time water simulation

Interaction with infrared sensing data

"Night style" artistic rendering

Simulation in first-person view of projection mapping event (used for content check or presentation)

Support of Oculus Rift virtual reality headset when available

System Developper

Development of an original media server synchronizating video on multiple computers

Tools for data preparation and network synchronization

36 screens (12x3) for a total resolution of 16392x2304

Technical support for interactive content on multiple screens

Design of the workflow for preparing contents

3D Studio Max deformer plugin: alter the shape of 3D model of animals to fit a custom surface

After Effects plugin for easy preparation of the mapping contents

3D Simulation software for preview

Automatic color calibration of the wall for 8 projectors

Real-time color correction

Design of the content workflow

Projection simulation

Design and development of the system of real-time 3D virtual character

Evaluation of several 3D Projection Mapping systems

Design of the reference content templates and development of an After Effects plugin for template transformation

3D Simulation software for preview

Research and Development, system design

Real-time cloth physics simulation

Kinect skeleton data filtering/retargeting, improvement of the body detection quality

Cloth database management system

Design Concept Graphic User Interface using both touch screen and Kinect motion sensing

Technical supervisor / Software Developper

Technical Supervision

Creation of a After Effects plugin for wall color correction

Technical Director / Interaction Designer

Multiple screens (8K pixels wide), double camera sensing

Real time drawing capture

Stroke path edition

Progressive stroke drawing

Object animation with spline interpolation

Real-time preview and high quality rendering

User interface developed using Qt

Technical Direction / Interaction Designer

Fluid and particle simulation

Real time drawing, edition and visualization

Importation of camera data from MAYA

User interface developed using MFC

Research and development about visual sensing algorithms used with standard camera (motion and position estimation, face detection and tracking)

Promotion (website, documents, presentation video editing)

Customer relations, project budget estimation

Design of interactive content by using Flash or C++/OpenGL

Development of customizable content templates and their IDE

System installation, calibration and maintenance

Development of an ActiveX Control performing camera capture, compositing and recording to MPEG-2 file

Creation of customized DirectShow filters

Programming of software adjusting automatically the volume amplitude of audio data (robust approach)

Evaluation of several video compression formats for optimal display on mobile phone

Programming of software/hardware/network diagnostic software

Prototyping of a new interface connecting two devices in order to decrease the system cost

This application was used in the context of the American TV series My Friends Tigger and Pooh

Digital measurements on 3D models

Scan anonymization

Scan stitching

Documentation, demonstration and training related with the software

Upgrade of the company 's website

Research and evaluation of methods especially in the following fields:

Processing of images and depth maps (Digital Elevation Model)

Scanner calibration, correction of the lens distortion

Mesh operations (polygonization, merging, simplification, comparison, positioning...)

Use of various data structures for optimization (octree, kd-tree...)

Reading volume data from DICOM files

Volume data processing

Programming of a volume rendering engine (real-time mode with texture stack approach, high quality mode with ray casting algorithm)

Polygonization of vascular network

Participation in the authoring of a software development normalization document

Software training

Calibration of a vision system

Construction of 3D mesh from profilometer data

Color correction

Projection/merging of 2D textures on the 3D mesh

Research about mesh construction algorithms, publication of an article