Introduction

Introducing Hyper-Skin, a uniquely designed hyperspectral dataset aiming to revolutionize hyperspectral skin analysis on consumer devices. With spatial and spectral resolution of 1024 × 1024 × 448, Hyper-Skin offers an extensive collection of hyperspectral cubes, providing over a million spectra per image.

A notable feature of Hyper-Skin is the inclusion of synthesized RGB images generated from 28 real camera response functions, enabling versatile experimental setups. What sets Hyper-Skin apart is its comprehensive spectral coverage, including both the visible (VIS) spectrum spanning from 400nm to 700nm and near-infrared (NIR) spectrum from 700nm to 1000nm, facilitating a holistic understanding of various aspects of human facial skin.

This dataset enables new possibilities for consumer applications to see beyond the visual appearance of their selfies and gain valuable insights into their skin's physiological characteristics, such as melanin and hemoglobin concentrations.

With the Hyper-Skin dataset, we aim to facilitate ongoing research in facial skin-spectra reconstruction on consumer devices, bringing affordable hyperspectral skin analysis directly to the consumer's fingertips.

Paper: Data Collection and Reconstruction Process

Our paper, presented at NeurIPS 2023, describes the comprehensive data collection process and the innovative reconstruction technique developed for this project. The main contributions of the paper are:

• Data Collection: A detailed description of the data acquisition methods used to collect high-quality hyperspectral images, including equipment setup, calibration, and preprocessing steps.
• Reconstruction Process: An in-depth explanation of the reconstruction methodology that translates RGB images into hyperspectral images. This section covers the underlying algorithms, optimization techniques, and model architecture used for this purpose.

Access the Paper

Data: Hyper-skin-2023 Dataset

The Hyper-Skin dataset consists of 306 hyperspectral data samples collected from 51 participants. Each participant contributed 6 images, covering 2 types of facial expressions and 3 different face poses. This diverse image collection ensures a broad representation of poses and facial expressions commonly encountered in selfies. The RAW hyperspectral data underwent radiometric calibration and were resampled into two distinct 31-band datasets. One dataset covers the visible spectrum, ranging from 400nm to 700nm, while the other dataset covers the near-infrared spectrum, ranging from 700nm to 1000nm. Additionally, synthetic RGB and Multispectral (MSI) data were generated, including RGB images and an infrared image at 960nm. The Hyper-Skin dataset comprises two types of data: (RGB, VIS) and (MSI, NIR), offering different needs in skin analysis and facilitates comprehensive investigations of various skin features.

---

Pair of (RGB, VIS) Data

The visible spectrum data in the Hyper-Skin dataset allows for the analysis of surface-level skin characteristics...

Example of (RGB, VIS) Pair:

---

Pair of (MSI, NIR) Data

The near-infrared spectrum data included in the Hyper-Skin dataset facilitates the study of deeper tissue properties...

Example of (MSI, NIR) Pair:

---

Dataset Attributes

The Hyper-Skin dataset offers a collection of 306 hyperspectral cubes, meticulously gathered from 51 unique subjects. Each subject’s facial skin is captured across three distinct angles (front, left, and right), and features two facial expressions (neutral and smile), ensuring diverse data to support robust facial skin analysis.

The hyperspectral images possess an impressive spatial resolution of 1024 × 1024 and capture 448 spectral bands. To facilitate targeted analysis, the data has been resampled into two 31-band datasets, one spanning the visible spectrum (VIS) from 400nm to 700nm and the other covering the near-infrared spectrum (NIR) from 700nm to 1000nm, both at 10nm steps.

Below is an example of the VIS and NIR spectrum graphs being generated from hyperspectral data:

Dataset Collection

Compliant with University research ethics protocols, the dataset collection process was carefully designed to ensure accuracy and reliability. The study was conducted under the approved protocol RIS-42284, and participants provided informed consent before data collection.

Data acquisition was performed using the Specim FX10 camera, a pushbroom camera known for its precision in hyperspectral imaging. The camera was mounted on a customized scanner to ensure precise scanning, and the subject's face was placed 40 cm from the camera. To enhance image quality, halogen lights were positioned on either side of the subject, providing optimal illumination.

Each hyperspectral image was captured with a frame rate of 45Hz for one line, taking approximately 22.7 seconds to scan all 1024 lines, covering the full range from 400nm to 1000nm. This meticulous approach ensured the high quality of the 448 spectral bands.

The dataset's demographic spans 51 participants aged between their 10s and 50s, with a gender distribution favoring males over females. The participants represent diverse ethnic backgrounds, including Asian, European, and Latino.

Code: Hyperspectral Image Reconstruction

We provide the complete codebase for reconstructing hyperspectral images from RGB images. The code includes scripts for data preprocessing, model training, and evaluation, as well as implementation of the algorithms described in our paper.

Objective

The primary objective of the provided code is to reconstruct hyperspectral images from standard RGB images efficiently and accurately. It includes:

• Data preprocessing pipelines.
• Model architectures and training scripts.
• Evaluation metrics and visualization tools.

Access the Code on GitHub

---

Citation

If you used the dataset for your publication, kindly acknowledge and cite the following work:

@inproceedings{ng2023hyperskin,
title={Hyper-Skin: A Hyperspectral Dataset for Reconstructing Facial Skin-Spectra from {RGB} Images},
author={Pai Chet Ng and Zhixiang Chi and Yannick Verdie and Juwei Lu and Konstantinos N Plataniotis},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2023},
url={https://openreview.net/forum?id=doV2nhGm1l}
}