Data from: Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence
Data files
Apr 02, 2024 version files 14.56 GB
-
Dataset1_CIFAR10.zip
-
Dataset2_MiniImageNet.z01
-
Dataset2_MiniImageNet.z02
-
Dataset2_MiniImageNet.zip
-
Dataset3_Omniglot.zip
-
Dataset4_BachChorales.zip
-
README.md
Abstract
The everlasting pursuit for a more powerful artificial general intelligence (AGI) continuously demands for higher computing performance. Breaking the restrictions from the Moore’s Law, integrated photonic neural networks have shown great potential in terms of superior processing speed and high energy-efficiency. However, the capacity and scalability are restricted by the unavoidable and time-varying on-chip errors that only simple tasks and shallow models (e.g., 10-category digits classification with thousand-parameter-level networks) can be on-chip realized optically. To support modern AGIs with optical computing, we innovate Taichi, large-scale photonic chiplets owning millions-of-neuron on-chip computing capability with 160 TOPS/W energy efficiency. For the first time, Taichi experimentally realized on-chip optical neural networks (ONNs) with over 13.96M neurons for thousand-category-level classification (tested 91.89% of accuracy for 1623-category Omniglot dataset) and high-fidelity artificial intelligence-generated content (AIGC) with state-of-the-art performances, while achieving up to 2 orders of magnitude improvement in computing efficiency than state-of-the-art AI chips. Based on an integrated diffractive-interference-hybrid design and a general distributed computing architecture, Taichi paves its own way for large-scale on-chip ONN models and advanced tasks, further exploiting the flexibility and potential of optical devices, which will lead to a giant leap in scalability, accuracy and efficiency to support modern AGI.
README: Data from: Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence
https://doi.org/10.5061/dryad.m63xsj497
Including the following datasets:
Dataset 1: CIFAR-10 dataset with image size 80 by 80 pixels.
Dataset 2: Mini ImageNet dataset for 100 category classification. We chose 100 categories from the original total of 1000 categories in the ILSVRC2012 ImageNet dataset. All the images in this modified dataset are in a resolution of 64 by 64 pixels. The category ID of the selected 100 categories are: 22, 29, 30, 33, 42, 48, 54, 72, 94, 101, 104, 119, 125, 135, 147, 148, 163, 168, 175, 194, 197, 206, 242, 245, 246, 247, 258, 273, 281, 291, 302, 309, 318, 323, 333, 334, 338, 363, 367, 371, 382, 397, 425, 431, 434, 445, 461, 463, 466, 477, 491, 492, 499, 501, 506, 527, 536, 557, 561, 570, 579, 585, 593, 608, 612, 617, 621, 629, 631, 648, 690, 697, 732, 733, 741, 748, 759, 766, 768, 788, 789, 791, 799, 816, 841, 852, 862, 869, 875, 885, 907, 911, 922, 927, 932, 936, 941, 953, 982, 990. The IDs are identical with the original ILSVRC2012 ImageNet dataset.
Dataset 3: Omniglot character dataset. The original one is presented in 'https://github.com/brendenlake/omniglot', which is a dataset containing 1623 categories.
Dataset 4: Bach Chorales dataset. The original one is presented in 'https://archive.ics.uci.edu/dataset/25/bach+chorales'. This dataset is used for training the content generation network.
The files in this repository are six seperate compressed folder for the four datasets. Please download and extract them for further usage.
For Dataset 1 (Dataset1_CIFAR10.zip), you will find four python numpy .npy format file containing the training and testing data along with their ground truth data. Using numpy.load() function to read them.
For Dataset 2 (Dataset2_MinilmageNet.zip, Dataset2_MinilmageNet.z01, Dataset2_MinilmageNet.z02), you will find a seperate validation data file, a folder for all training data (in 10 seperata files), and a .py script to load these data to your programme. The required python packages for this script are listed in the beginning lines in the script itself. (Note that you need to download all the three .zip files and then extract them together.) Please change the file path in the python script to match your file structure.
For Dataset 3 (Dataset3_Omniglot.zip), you will find a folder containing the character images and character names, and a seperate .py script to load the images. The data structure in the extracted 'Omniglot_Full' folder goes like 'dataset name - language name - character sequence - images for this character'. The attached python script will automaticlly read the entire dataset and form the 'input data - ground truth' pairs for network training. Please change the file path in the python script to match your file structure.
For Dataset 4 (Dataset4_BachChorales.zip), you will find a folder containing 413 clips of original Bach chorales in .mxl format. These files can be processed using a variety of music processing softwares or python callable packages such as MuseScore. The notes in each music chip could be mapped to a series of vectors to fit the neural network scale.
Even though the datasets mentioned in this repository are all public dataset, the actual files for them are slightly different from the original one. The image or music clip sizes are modified and unified to match our photonic chip tests. Please cite the original paper for each dataset as well as this repository if you considered to use these data in your research. Thank you very much.
I, the copyright holder of this work, hereby publish it under the following license: CC0 1.0 Universal (CC0 1.0) Public Domain Dedication.
Methods
The preprocessed dataset that modified from the original ones. We resized/cropped the dimension of the images and re-origanized the training-testing splitting in the original datasets, to fit the data requirements by our optical computing hardware Taichi chiplets.