The truth is that Yalta did not hand Eastern Europe to the Soviets. That territory was already in their possession. Stalin had made clear his plan to take over as much territory as possible back in the Molotov-Ribbentrop pact of 1939, which carved Poland in half and gave the Soviets the Baltic states. The discovery in 1943 of the massacre of Polish officers by the Soviet army in the Katyn forest was further evidence of Stalin's malign intention to exterminate the leadership of Poland. Then, in 1944, during the Warsaw uprising by the Polish Home Army, Stalin halted the advance of his army on the banks of the Vistula River and allowed Nazi SS units to return to slaughter the Poles. By the time of Yalta, the Red Army occupied all of Poland and much of Eastern Europe.
Theoretically, Churchill and Roosevelt could have refused to cut any deal with Stalin at Yalta. But that could have started the Cold War on the spot. It would have seriously jeopardized the common battle against Germany (at a moment when Roosevelt was concerned with winning Soviet assent to help fight the Japanese, which he received).
Supreme Allied Commander Dwight Eisenhower was happy to let the Soviets bear the brunt of the fighting as they marched toward Berlin, and he was unwilling to expend American troops on storming the German capital. The only one who was eager to do that was Gen. George Patton, who hoped to take on the Russians as well. Given the domestic pressure to "bring the boys back home," Roosevelt would have been taking a politically suicidal course had he broken with our allies, the Soviets.
Roosevelt was hardly perfect at Yalta. He was naive about Stalin's intentions and believed he could cajole the dictator into following more moderate policies. But FDR's approach was not particularly different from that of Churchill (who had declared that he would "sup with the devil" to win the war, which is what he and Roosevelt, in effect, did).
As for the charges about Hiss' influence, they've been overblown by the right for political purposes; in fact, Hiss was a minor player at Yalta.
What's more, it was the isolationist right that never wanted to fight the war in the first place, which it conveniently forgot once it began attacking Democrats as being soft on communism. Nothing of course could be further from the truth. Roosevelt went on to recognize Stalin's perfidy shortly before he died, and it fell to Truman to fight the Cold War.
Roosevelt's record is no cause for shame, but Bush's comments are.