In this Letter we show that the vortex lattice structure in the Bose-Fermi superfluid mixture can undergo a sequence of structure transitions when the Fermi superfluid is tuned from the BCS regime to the BEC regime. This is due to the difference in the vortex core structure of a Fermi superfluid in the BCS regime and in the BEC regime. In the BCS regime the vortex core is nearly filled, while the density at the vortex core gradually decreases until it empties out in the BEC regime. Therefore, with the density-density interaction between the Bose and the Fermi superfluids, interaction between the two sets of vortex lattices gets stronger in the BEC regime, which yields the structure transition of vortex lattices. In view of the recent realization of this superfluid mixture and vortices therein, our theoretical predication can be verified experimentally in the near future.