Always helps to know what the vehicle is to keep assumptions to a minimum. If it's a sii, then yes, access to the shaft seal is very restricted. If this is a fullsize pick-all, access is much more generous. It can be removed from above in an sii with an assortment of picks and hooks to jerk it out. It makes for an extremely tedious and cuss-laden endeavor. At one time, you could by a tool that was made specifically for removing these seals from above. It threaded down into the metal clad portion of the seal engaging the entire circumference. Then all you did was pull on that tool and the seal would pop right out with it. They were damn spendy for something the average person might only use once in their life, but worth every penny if still available. I think they're tough to come by these days.
Since you're planning on replacing the pan gasket too, that means the pan will be off and the fluid will be drained. Its a good time to throw on a fresh filter and verify band tension is within spec. With the pan removed, the valve body can then be removed, which would then allow you to pound the shaft seal out from below. This is much quicker and easier than trying to pry it out from above. The new seal would then get driven in from above. Now, vb removal has a few ticklish steps to it as well, so I can understand your trepidation, but having gone at this seal both ways, and having done several vb removals, its a no-brainer for me. If I have a failed seal to remove in a sii, I'm draining fluid, dropping pan and yanking vb to pop the seal out from below. But that's just me. You might try prying it out from above and land on just the right combination of picks and mini-pry tolls that you get the damn thing popped out in short order. That just hasn't been my experience.