Many cars have check valves built in the fuel pump to keep fuel pressure constant even when the car is turned off. This helps the car start right away when sitting for a long time. It is normal to have pressure and I don't expect that is your problem. Gas fumes are often a sign of a gas tank venting system failure. Cars will have a carbon canister "evap canister" to catch fumes from the tank. The vent line between the gas cap and the carbon canister may be clogged. The tank fuel or vent lines may be damaged. The fittings at the gas tank may be rusted through. Your tank may be leaking. The vapor canister check valves may be malfunctioning.
This guy has a great way to test the evap system.