You didn't say weather or not you had a catch bottle so I'll assume not. If this is the case, the expansion of the coolant overflows out the tube but as it cools air is likely to get sucked back in. Overflow bottle type cooling systems use a rad cap that allows fluid to be drawn back into the cooling system as the engine cools down. The froth is probably just that, coolant whipped up with air in it, sitting at the top of the upper tank mixed with rusty engine crud or sealant like Bars Leak. Next time this happens, let the engine cool completely and then remove the cap on the rad an see what level the coolant is at. It should be right to the top of the tank (bottom of the filler neck). If the presence of air is causing the whipping action, best invest in an overflow tank and the matching rad cap. Air is the worst enemy of engine cooling systems.