What happens if I delete a field in the new field management page that exists in a FER. Does it remove it from the FER?

You can't delete Field from the field management page until you remove the reference from the Field Extraction Rule (FER).

Deleting a field does not delete historical data assigned to that field. If you delete a field by mistake and one or more of those dependencies break, you can re-add the field to get things working properly again.

You should always disable a field and ensure things are behaving as expected before deleting a field. Please follow this link to get more information on deleting a field from the new field management page.