The tube the pushrod is in the tube ABOVE the cavity for the fuel pump, however, the actual pushrod is too long to go into the tube from the cavity's opening, so the tube "continues" below the cavity and is closed with a pipe plug. When the push rod is installed or replaced, the plug is removed and the push rod can be removed or inserted through that hole. It goes up into the tube above the cavity and if the friction is just right, it will be held up in place and the fuel pump and its arm will just bolt on.
However, if the camshaft nub that pushes the pushrod down against the fuel pump's arm is down and pushing the pushrod down, it may not have enough friction to stay up, so many use think grease to hold it up.
In the picture above, you can see that the push rod is against the arm - that is ABOVE the cavity in the block. The pipe plug you see below the cavity would be on the left side of the picture above the words FUEL SUPPLY and would allow installation and removal of the push rod. The push rod can't fall out of the block if the plug and/or the pump are in place, but if the pump has been removed and it falls down within the cavity as it often does, it will interfere with the installation of the pump.
If you can't feel it, it's likely in the up position, and you got lucky with camshaft positioning. Install the pump and you're good to go.
