The visitors batted much better second time around but still faced defeat against Australia A