In the internal dose evaluation, the specific absorbed fraction (SAF) and S-value are calculated from the reference phantom based on Caucasian data. The differences in height and weight between Caucasian and Asian may lead to inaccurate dose estimation. In this study, we developed the Taiwanese reference phantoms. 40 volunteers were recruited. Magnetic resonance images (MRI) were obtained, and the contours of 15 organs were drawn. The Taiwanese reference man (TRM) and Taiwanese reference woman (TRW) were constructed. For the SAF calculation, the differences in the self-absorption SAF (self-SAF) between the TRM, TRW, and Oak Ridge National Laboratory (ORNL) adult phantom were less than 10% when the difference in organ mass was less than 20%. The average SAF from liver to pancreas of TRM was 38% larger than that of the ORNL adult phantom, and the result of TRW was 2.02 times higher than that of the ORNL adult phantom. For the S-value calculation, the ratios of TRW and ORNL adult phantom ranged from 0.91 to 1.57, and the ratios of TRM and ORNL adult phantom ranged from 1.04 to 2.29. The SAF and S-value results were dominantly affected by the height, weight, organ mass, and geometric relationship between organs. By using the TRM and TRW, the accuracy of internal dose evaluation can be increased for radiation protection and nuclear medicine.