The book is set in Vienna on one day around the year of its publication, 1985. (p. 193) Reger is an 82-year-old music critic who writes pieces for The Times. For over thirty years he has sat on the same bench in front of Tintoretto's White-bearded Man in the Bordone Room of the Kunsthistorisches Museum for four or five hours of the morning of every second day. He finds this environment the one in which he can do his best thinking. He is aided in this habit by the gallery attendant Irrsigler, who prevents other visitors from using the bench when Reger requires it.
The book is narrated entirely by Atzbacher, who met Reger in the museum the day before and with whom Reger then arranged to meet again in the museum on this day - thus, exceptionally, visiting the museum on two consecutive days. They had arranged to meet in the Bordone Room at 11.30, but they both arrive early, and the first 170 pages of the book consist of Atzbacher's thoughts and recollections as he surreptitiously watches Reger in his usual position. These are dominated by Reger's thoughts and recollections, as previously related to Atzbacher. Atzbacher tells of the deaths of Reger's wife and sister, and of his contempt for various aspects of Austrian and occasionally German society, including Stifter, Bruckner and Heidegger, the state and "state artists" in general, and the sanitary condition of Viennese toilets. Reger considers the idea of a supposed "perfect" work of art to be unbearable, and so seeks to render them bearable by finding flaws within them.
The second half of the book, once Atzbacher and Reger have met, is formed of the intertwined reports of Reger's speech now, in the museum, with what he had earlier said at a meeting of the two in the Ambassador hotel after his wife's death, and his statements when they had met in Reger's flat before her death. This death of Reger's wife - its circumstances and its effects on him - increasingly dominate the book as it moves towards its conclusion. It is revealed that Reger had first met his wife while sitting on the Bordone Room bench, and that she had then accompanied him on his visits to the museum. It was while walking there in winter that she had suffered an ultimately fatal fall, for which Reger blames the town authorities (for failing to maintain the path), the state (the owner of the museum, which failed to provide timely aid), and the Catholic church, which runs the Merciful Brethren Hospital which Reger believes botched an operation which could have saved her.
Despite his continued attacks on the "Catholic National Socialist" museum and state (p. 301) and his contempt for humanity, exemplified by the conduct of his housekeeper in taking advantage of him after his wife's death, Reger describes how he overcame his initial inclination to suicide and managed to survive her. He found himself let down by art, which proved useless to him at the decisive moment:
"no matter how many great spirits and how many Old Masters we have taken as companions, they can't replace any people, said Reger, in the end we are abandoned by all these so-called great spirits and by these so-called Old Masters, and we see that we are mocked in the meanest way by these great spirits and Old Masters". (pp. 291-2)
Convinced that people are the only possible means of survival, Reger re-engages with the world, aided only by his "misuse" of Schopenhauer (p. 288) and by the White-bearded Man, the only work in the museum to have stood up to his scrutiny for thirty years.
The book concludes with Reger revealing the true purpose of his arranging to meet Atzbacher: to invite him to a performance of "The Broken Jug" that evening, despite his own hatred for drama. Atzbacher accepts, reporting that "the performance was terrible".