"Talk about a place of hard knocks," he said. "I know where all of them are."
Walker moved to a small apartment near Columbia University when he was 9 to live with his brother Joe, who was 25 years older, and his family. When asked about his role models, Walker named Churchill, Martin Luther King, Jackie Robinson and the late Joe Walker, who was not formally educated beyond junior high but had "a doctorate in common sense."
Joe Walker also had good business sense, owning three barbecue restaurants in Harlem and a glass cutting and window cleaning service. Early in the days when government agencies were required to hire a certain percentage of minority contractors when possible, he had one of the few window cleaning businesses owned by a black. As a result, he and his crew, including LeRoy, were offered jobs throughout the East. A regular client was the Pentagon.
"I've been hanging out from 30 or 40 stories with nothing between me and death but a belt," Walker said. "But it was fun. I kept my union card until I had my doctorate. I used to say, 'I'm going to keep this card because if anything goes bad at the university, I've got a trade.' "
Joe Walker taught his brother business administration through the restaurants, one of which was next door to boxer Sugar Ray Robinson's bar. By the time he went to college, LeRoy Walker was so involved in the operation that Joe gave him a percentage of the profits.
"We used to have people coming from downtown in limos to get our spit-cooked barbecue," Walker said. "I was one of the few people in college whose mother used to write me for money because I did so well."
But Joe was not all business. He also made sure LeRoy was exposed to culture, once insisting that he accompany clients whose house they were painting to the Metropolitan Opera to see "Aida."
"I didn't understand one word they were singing," Walker said. "But that's how I was inspired to learn about languages."
He studied French and German at Benedict College.
LeRoy was more in his element during the summer between his junior and senior years of high school when Joe arranged for him to work for a swing band after a valet had stolen bandleader Jimmy Lunsford's horn and pawned it. LeRoy was responsible for making sure uniforms were clean, pressed and delivered on time, that the instruments were safe and that the band had food and coffee.
The early jobs that summer were at the Cotton Club and the Lafayette, where Duke Ellington ruled, but Lunsford's band later went on the road, where Walker was shocked by the racism he found. Even the White Castle hamburger stands in some cities would not serve the band.
"That was the first time racism really hit me," he said. "I lived in the South until I was 9, and when I would go into a department store there would be two water fountains, one for colored--which blacks were called then--and one for whites. And there were worse things than that. But that was the mores of the South then. Yeah, it hurt, but, at least, you knew where everybody stood.
"But it was in places where the racism was more subtle, like in the North in those days, where it really got to me. I could go on and on with stories, like the time I had to go to five different hotels in Philadelphia before I found one that would let me have a room. There, in the City of Brotherly Love. I don't think people understand how deeply some of these things are ingrained in me."
Years later, as track coach at North Carolina Central, Walker stressed two messages to his athletes.
One is that nothing in life comes easily, which is why he emphasized academics as well as athletics. In 30 years, only 12 of his athletes failed to graduate.
The other is that individuals should not be judged by their skin color.
"I felt so bad for my athletes because of some of the things we had to go through," he said. "Sometimes we had to ride hundreds of miles before we could find a restaurant--not to go into to eat but (one) that would even serve us sandwiches to take out.
"But I always told my athletes that they shouldn't make any unilateral judgments about a group of people."
As the times became more militant, however, his conciliatory efforts were not always appreciated, such as during the 1968 Summer Olympics at Mexico City. His longtime friend, Jesse Owens, asked Walker to join him in speaking to U.S. athletes who were planning demonstrations to protest racial discrimination at home.
Owens was Walker's boyhood idol, and he dreamed of succeeding the hero of the 1936 Summer Olympics as the 100-meter champion. It was not entirely a fantasy, because Walker was fast enough to have made the U.S. team, but neither he nor the rest of the world at that time could outrun the Nazis, who forced the cancellation of the 1940 Summer Games by starting a war.
