By Jim Finkle
Palmisano will remain chairman after nearly a decade as the company's chief executive, during which he helped transform Big Blue from a personal computing hardware company into a global services behemoth.
Rometty, 54, takes over on January 1. Shares in IBM slipped 0.7 percent after hours.
"Given Ginni's experience running the largest portion of the business by revenue, she was a logical choice," said Macquarie Securities analyst Brad Zelnick.
In nine years as IBM's leader, Palmisano, 60, exited low-margin businesses including PCs, printers and hard drives. He expanded the company's offerings in services, consulting and software.
Rometty most recently served as senior vice president of global sales for IBM after leading its global services business, where the company is the world's biggest provider of technology services.
"She has done well at IBM. She has contributed to their expansion overseas -- emerging markets -- and has done a fantastic job in that space," said Morningstar analyst Sunit Gogia. "All the public knowledge about her performance is very encouraging."