Voxel phantoms developed by segmenting computed tomography images are known to be more anatomically accurate than mathematical phantoms. However, due to their lack of flexibility and the complexity of voxel datasets, the use of voxel phantoms in dosimetry is often impractical. This paper describes the development of the realistic anthropomorphic flexible (RAF) polygonal mesh phantom, a novel phantom based on Boundary Representation (B-Rep) that merges anatomical accuracy and flexibility. Rather than using segmentation of tomography images, the modeling of the phantom’s organs was based on freely and commercially available anatomical atlases, ICRP 89, and recent medical literature. To validate the phantom, a high-resolution voxel version was created for the MCNPx transport code. The voxelized RAF phantom was validated by comparing it with the ICRP 110 male phantom for external irradiations with parallel beams of photons and electrons. Dose coefficients obtained from simulations with the RAF phantom were compared with those from ICRP Publication 116. Relevant differences in organ doses were found.