According to leader-member exchange (LMX) theory, developing differentiated LMX among team members is an effective managerial practice for team leaders to promote work efficiency and team performance; however, little is known about how leaders establish LMX differentiation (LMXD) and its subsequent impacts on leaders themselves. Building on LMX theory, this study explores whether and when LMXD is shaped by members’ variabilities in in-role performance as well as leader-directed organizational citizenship behavior (OCB) within a team. We further propose that leader-leader exchange (LLX) plays a different role in moderating the relationship between these team-level variabilities and LMXD. Finally, we argue that LMXD has a negative effect on members’ perceived leadership effectiveness. Data from 1178 team members and 209 team leaders supported our model. Results showed that both team members’ in-role performance variability and leader-directed OCB variability had a positive effect on LMXD. The positive relationship between team members’ in-role performance variability and LMXD was stronger for leaders with low levels of LLX. In contrast, the positive relationship between team members’ leader-directed OCB variability and LMXD was stronger for leaders with high levels of LLX. Furthermore, LMXD was negatively related to leadership effectiveness. This study contributes to a more complete theorization of the formation and leadership outcomes of LMXD.