The arguments of anti-Mormons regarding Joseph's wives tend to rely on 21st century sensibilities regarding age and age differences, with the assumption that these polygamous marriages involved sex and were all about sex. A reasonable response to these arguments and assumptions is found in new "wiki" entry in the FAIR Wiki by FAIRLDS.org , "Joseph Smith's Marriages to Young Women." I recommend this article. It deals specifically with the two most controversial cases of young wives, Helen Mar Kimball, who was 14 when married, and Fanny Alger, who was 16. It also provides perspective on the issue of age differences in marriages in that day.
Women can legally marry at age 16 right now in Wisconsin and many other states (typically parental consent is needed), and I've known women who were married at that age (and not just Hmong women, where marriages from age 13 to 16 are surprisingly common in Wisconsin). It's the age 14 marriage to Helen Mar Kimball that is especially troubling, but as Todd Compton points out, "there is absolutely no evidence that there was any sexuality in the marriage, and I suggest that, following later practice in Utah, there may have been no sexuality. All the evidence points to this marriage as a primarily dynastic marriage."
While I don't "get" polygamy, and am grateful not to have it be a part of our culture today, I think we need to be careful about condemning those who practiced it in the past. The evidence does not point to sex as the driving force for that practice, and it may not have even been involved in many of the marriages.
When it comes to discussing the evidence of authenticity for the Book of Mormon, I do not consider allegations of lechery in Church leaders to be a reasonable response, though it has all too often been a diversionary tactic. If the Book of Mormon is true, then something divine happened between 1820 and 1830, and that needs to be accounted for, even if one is troubled or even outraged by polygamy.
His enemies have written many volumes condemning Joseph Smith, but those who knew him best, those of strong Christian heritage, did not find a lecher or pedophile in their midst - they found a prophet of God teaching and striving to live a pure religion. While polygamy leaves many questions unanswered and may have been implemented poorly many times, I think it would be a terrible mistake to let it be the deciding factor that let someone give up on the restored Gospel of Jesus Christ.
For more information, here's an excerpt from my March 18, 2006 post:
Gregory L. Smith, M.D., offers a helpful and highly informed perspective on the controversial 19th-century practice of polygamy among Latter-day Saints in his article, "Polygamy, Prophets, and Prevarication: Frequently and Rarely Asked Questions about the Initiation, Practice, and Cessation of Plural Marriage in The Church of Jesus Christ of Latter-day Saints" at FAIRLDS.org. If you've struggled with the issue of polygamy (struggled with it philosophically, not in practicing it, I hope!), this article helps clarify why it was kept secret for so long, why it was not about personal ratification, etc.