This Letter presents a construction of isospectral problems on the torus. The construction starts from an SU(n) version of the XYZ Gaudin model recently studied by Kuroki and Takebe within the context of a twisted WZW model. In the classical limit, the quantum Hamiltonians of the generalized Gaudin model turn into classical Hamiltonians with a natural r-matrix structure. These Hamiltonians are used to build a nonautonomous multi-time Hamiltonian system, which is eventually shown to be an isomonodromic problem on the torus. This isomonodromic problem can also be reproduced from an elliptic analogue of the KZ equation for the twisted WZW model. Finally, a geometric interpretation of this isomonodromic problem is discussed in the language of a moduli space of meromorphic connections.