US Navy secretary reacts to criticism with lunch invite