Fix check of rhs of type family instances (#2157)