My '67 Newport does similar. On the first stop, I do it easily as it can pull to the right for the first stop or two.
I never have taken it apart as it goes away and does not return until it has sat several days between driving. I'm suspecting it is "lot rot", as in a thin layer of rust on the brake drum friction surface. Until the rust gets wiped away, the rougher surface grabs a bit. If I drive it daily, no problems, but let it sit several days . . .
You can probably chunk parts at it and not really help the situation, so until something breaks or leaks, it can be something to deal with. Be sure to advise any other people who might drive the car of the issue, too. They might think "It's just an old car . . . "
You might lube and adjust the front wheel bearings for good measure?
Just my experiences,
CBODY67