We do not have to view Modal, Conditional, or Unreal Past patterns as separate. With Perfect tenses, our syntactic HAVE helps tell about real time. It has an open real-time frame. With the Unreal Past or Conditional, HAVE tells about hypothetical time. HAVE is not part the real map, then. It comes with an auxiliary compass for relative time.
We can view verb forms as generally relative, for hypotheses. PRESENT forms can tell about the FUTURE. PAST forms can tell about the PRESENT. ANTECEDENT PAST forms can tell about the PAST.