FGS (Fine Granularity Scalability) is a scalable coding technique which can provide flexibility and good performance for Internet video streaming. However, FGS is not suitable for wireless video streaming. This is mainly because the low coding efficiency of FGS does not fit the limited bandwidth of wireless networks. In this paper, we jointly consider mode selection and UEP (unequal error protection) for FGS video transmission over wireless channels. In particular, we provide two modes for encoding the FGS enhancement layer of each video frame, i.e., with prediction or without prediction. The mode selection depends on the capability of UEP while the solution for UEP depends on the vulnerability of source data. We construct an overall end-to-end rate-distortion (R-D) function. Based on this end-to-end R-D function, we are able to find the optimal solutions for both mode selection and UEP so that an optimal tradeoff between efficiency and robustness can be achieved. Experimental results demonstrate the proposed system is able to significantly improve the end-to-end video quality for wireless FGS video coding and transmission.