Per the FSMs,
Clean all parts in gas or solvent. Blow out the orifices, pump nipple passage and nipple valve spring guide with compressed air.
Examine the check valve nipple surface and threads for damage or debris that would hinder the ball from seating or operation of the spring.
Inspect the ball spring for breakage or rust. Check the free length of the spring.
Inspect the check ball for wear rings or rust.
Balls not perfectly round should be replaced.
Inspect the ball seat in the pump body using a light. Check for pitting or debris that would hinder the ball and seat.

Check the body and parts for wear or cracks.

The seal lip goes toward the feed gears.

^{1) 2)}

Click here for the 57-76 Oil Pump Damage and Repair page in the Sportsterpedia.