Background: Despite intensive efforts devoted to collecting human polymorphism data, little is known about the role of gene flow in the ancestry of human populations. This is partly because most analyses have applied one of two simple models of population structure, the island model or the splitting model, which make unrealistic biological assumptions. Results: Here, we analyze 98-kb of DNA sequence from 20 independently evolving intergenic regions on the X chromosome in a sample of 90 humans from six globally diverse populations. We employ an isolation-with-migration (IM) model, which assumes that populations split and subsequently exchange migrants, to independently estimate effective population sizes and migration rates. While the maximum effective size of modern humans is estimated at ∼10,000, individual populations vary substantially in size, with African populations tending to be larger (2,300-9,000) than non-African populations (300-3,300). We estimate mean rates of bidirectional gene flow at 4.8 × 10-4/ generation. Bidirectional migration rates are ∼5-fold higher among non-African populations (1.5 × 10-3) than among African populations (2.7 × 10-4). Interestingly, because effective sizes and migration rates are inversely related in African and non-African populations, population migration rates are similar within Africa and Eurasia (e.g., global mean Nm = 2.4). Conclusion: We conclude that gene flow has played an important role in structuring global human populations and that migration rates should be incorporated as critical parameters in models of human demography.