Test for (fd < 0) before trying to FD_SET it