6DOF pose estimation - synthetically generated dataset using BlenderProc
Data files
Nov 26, 2023 version files 40.60 GB
Abstract
Accurate and robust 6DOF (Six Degrees of Freedom) pose estimation is a critical task in various fields, including computer vision, robotics, and augmented reality. This research paper presents a novel approach to enhance the accuracy and reliability of 6DOF pose estimation by introducing a robust method for generating synthetic data and leveraging the ease of multi-class training using the generated dataset. The proposed method tackles the challenge of insufficient real-world annotated data by creating a large and diverse synthetic dataset that accurately mimics real-world scenarios. The proposed method only requires a CAD model of the object and there is no limit to the number of unique data that can be generated. Furthermore, a multi-class training strategy that harnesses the synthetic dataset's diversity is proposed and presented. This approach mitigates class imbalance issues and significantly boosts accuracy across varied object classes and poses. Experimental results underscore the method's effectiveness in challenging conditions, highlighting its potential for advancing 6DOF pose estimation across diverse applications. Our approach only uses a single RGB frame and is real-time.
README: Data Repository README
This repository contains data organized into a structured format. The data consists of three main folders and two files, each serving a specific purpose.
The data contains two folders - Cat and Hand.
Cat Dataset: 63492 labeled data with images, masks, and poses.
Hand Dataset: 42418 labeled data with images, masks, and poses.
Usage: The dataset is ready for use by simply extracting the contents of the zip file, whether for training in a segmentation task or a pose estimation task.
To view .npy files you will need to use Python with the numpy package installed. In Python use the following commands.
import numpy\
data = numpy.load('file.npy')\
print(data)
What free/open software is appropriate for viewing the .ply files?\
These files can be opened using any 3D modeling software like Blender, Meshlab, etc.
Camera Matrix Intrinstics Format :
Fx 0 px
0 Fy py
0 0 0
Below is an overview of the data organization:
Folder Structure
- Rgb:
- This folder contains a collection of synthetically generated images.
- Images are organized based on index numbers.
- Example:
rgb/0.png
,images/1.png
, ...
- Mask:
- This folder contains corresponding masks for the images.
- Masks are also organized based on index numbers.
- Example:
mask/0.png
,masks/1.png
, ...
- Pose:
- The poses folder holds pose data associated with the images.
- As with the images and masks, poses are also indexed.
- Example:
pose/0.npy
,pose/1.npy
, ...
Files
- 3D Model:
- This repository includes a 3D model file.
- The 3D model is named
model.ply
.
- Intrinsic Parameters:
- A text file named
camera.txt
is provided. - This file contains intrinsic parameters for the data.
- Intrinsic parameters are essential for various computer vision tasks.
- A text file named
Usage
This following model can be directly used to train a 6-DOF Pose Estimation model using Clean-Pvnet
Acknowledgments
This dataset was created and processed using BlenderProc, a powerful tool for generating synthetic data in Blender. We would like to express our gratitude to the BlenderProc development team and community for their invaluable contribution to the generation and manipulation of 3D data.
Citation
If you use this dataset in your work, please consider citing BlenderProc as follows:
M. Denninger et al., “BlenderProc: Reducing the Reality Gap with Photorealistic Rendering,” Journal of Open Source Software, Jul. 2020, [Online]. Available: https://elib.dlr.de/139317/
Methods
This dataset has been synthetically generated using 3D software like Blender and APIs like Blendeproc.