‘I think if he felt people were fussing over him he could be quite outspoken about that,’ the author said