BREAKING BARRIERS by Carl T. Rowan (Little, Brown and Company: $22.95; 372 pp.) . Growing up African American in the Deep South, Carl Rowan was once so poor that he found food by sneaking into farms and "sucking hot milk from the teats of a cow." He nevertheless managed to become an official in the Kennedy and Johnson Administrations, and in this memoir, he offers portraits of the fundamentally decent Americans who helped him that recall the golden-hued memories of his nemesis, Ronald Reagan. Unlike Reagan, however, Rowan realizes that millions of other Americans did not enjoy his luck; he was able to escape his town's plunge into abject poverty, for instance, when the Navy sent recruiters to his small college, mistaking it for the University of Tennessee. "We act as though it is only in distant places like Bucharest and the Baltics, Panama and Prague, that ordinary people have a terrible time finding hope and happiness," he writes. "I know better." "Breaking Barriers" is a thoroughly eloquent work, marred only by Rowan's failure to reconcile his vigorous defense of the social programs he helped champion, now under attack from writers such as Shelby Steele and Jim Sleeper, with his assertions that African Americans want "no special restrictions and no special favors; just the right to rise or fall on merit."