Vector field electron tomography (VFET) reconstructs vector fields based on phase maps recorded from two or more orthogonal tilt series. The tomographic reconstruction of vector fields involves considerations beyond those involved in the reconstruction of scalar fields. Here we examine the effect of initial magnetization orientation on reconstruction errors. The orientation of a magnetic particle affects the contrast in the phase maps. This, in turn, affects the accuracy of the reconstructed vector fields. We derive expressions that model the dependence of reconstruction errors on initial specimen orientation when using a filtered backprojection algorithm to reconstruct a vector potential from two tilt series. We compare these analytical results with those from numerical simulations. Our results can inform experimental procedures, such as sample preparation techniques and the choice of tilt series orientations. Specimen orientation can be a significant source of error in VFET, and our results can provide the means to minimize these errors.