For numerical approximation the reformulation of a PDE as a residual minimisation problem has the advantages that the resulting linear system is symmetric positive definite, and that the norm of the residual provides an a posteriori error estimator. Furthermore, it allows for the treatment of general inhomogeneous boundary conditions. In many minimal residual formulations, however, one or more terms of the residual are measured in negative or fractional Sobolev norms. In this work, we provide a general approach to replace those norms by efficiently evaluable expressions without sacrificing quasi-optimality of the resulting numerical solution. We exemplify our approach by verifying the necessary inf-sup conditions for four formulations of a model second order elliptic equation with inhomogeneous Dirichlet and/or Neumann boundary conditions. We report on numerical experiments for the Poisson problem with mixed inhomogeneous Dirichlet and Neumann boundary conditions in an ultra-weak first order system formulation.