Modeling and simulation is an important activity in the development of complex control systems such as the multi-robot formation control. This study addresses the use of state based approach to model the complex systems that are employed to perform tightly coupled tasks in an unknown environment. The proposed state based model represents the entire behavior of the complex system by a global state machine or Augmented Finite state machines, which describes the complete intra-object behavior of the system. The methods to model and implement the state machines in the MATLAB-simulink/State flow environment are also addressed in this paper. Further, performance of the modeling approach is investigated through simulation studies by considering the multi-robot formation combined with navigation and role switching and are addressed in this study. The simulation result suggests that the state based model and simulation based on Stateflow could be a viable method to model mutli-domain systems such as the highly complex robotic systems with visualization, performance and formalism.