Extracting diamonds: Identifiability of 4-node cycles in level-1 phylogenetic networks under a pseudolikelihood coalescent model

Abstract

AbstractHere, we investigate the (generic) identifiability of the position of the hybrid node in a 4-node hybridization cycle in a semi-directed level-1 phylogenetic network. While generic identifiability is easily attained under non-restrictive assumptions such ast∈ (0, ∞) for all branches andγ∈ (0, 1) for the inheritance probability of the hybrid edges, simulations show that accurate detection of these cycles can be complicated by inadequate sampling, small sample size or gene tree estimation error. We identify practical advice for evolutionary biologists on best sampling strategies to improve the detection of this type of hybridization cycle.