A retrieval technique for crystal structures using high-resolution electron microscopy is presented. The inversion of the complex structure-to-image relation is performed by numerical optimization of the configuration space of object models. Unlike structure refinement in X-ray crystallography, the method operates on unknown defect structures on a nanometre scale. The diversity of crystal defects examined and the differences in microscope types and alignment conditions makes it necessary to adapt the basic algorithm to a broad variety of needs resulting in a modular program package for general use. New developments, such as a real space slice function calculation, problem adapted optimization strategies, and finally an examination of iterative matching of image series and exit wavefunctions are presented.