Justice Michael Kirby was jet-lagged. He was just off the red-eye after a trip overseas for UNAIDS work and speaking engagements, and small mountains of correspondence waited on a side table. Paper covered his desk and rose in piles from the floor around it.

"Come around here," he said.

"I won't tread on the paper," I replied.

"Oh, you can tread on it. Now, what do you think of this?"

He was pointing at a piece of paper. I leaned over his shoulder to look, but it was a ruse: the judge's other associate raised a camera and snapped our photo. "One for the album," said the judge. "Not like the photo of me they always run in the Australian. I mean, I know I'm no box of chocolates to look at, but the one they run is really terrible."

From the windows that ran the length of the room stretched a clear view across the boat-flecked harbour to the Sydney Heads and the horizon. Competing for attention, on the wall opposite, were the judge's prized photographs, framed and mounted for display.

Photography would turn out to be central to my role as Justice Kirby's associate. I would take countless photographs, all to be stuck into large black albums at year's end, during the quiet weeks of the court's summer recess. And I would sometimes bring out my own camera, with the judge exclaiming that I used film, which was very old-fashioned.

That first day, we went out for lunch with one of the judge's old associates. "James rang and said that he had booked a table away from the windows," Kirby said. "I told him that I wanted to sit in front of the windows." As we went down the judges' lifts and came out into Phillip Street and a dazzling Sydney day, the judge donned a pair of round sunglasses. "I'll wear these for James's benefit," he said. "He'll enjoy the mystery of it."

Kirby has remained one of what he calls the "ordinary folk"; he has never acquired a taste for the fine foods that a judge's salary can afford. Once, he rang chambers from Los Angeles, on his way home from a law-reform conference in Dublin. "Johan and I are off to our old haunts," he said. "We'll have breakfast at a diner and later we'll go to Sizzler."

Nor is he much of a drinker. A prim Protestant, he would call himself, in accepting just "a thimbleful" of wine at the lunches he hosted in his chambers. When these lunches concluded, leftovers were saved. "I put those strawberries in a piece of plastic," Kirby would say, placing them in a plastic bag to carry off home.

Kirby's wining and dining of visitors at such events has not been about indulgence, but an effort to engage, to cultivate. The same effort is present in his writings ­­- not only his judgements, but his speeches and letters. A profligate correspondent, Kirby sends dozens of letters a week, ferreting things out and firing them off: "I enclose herewith [such and such], which you may find useful in your endeavours."

It has been this effort to engage, as much as the burden of dissent, that has kept Kirby far busier than his contemporaries on the court. He would spend weekends holed up in chambers, reading and dictating, his gravelly voice audible from the corridor as he poured out a judgement for a criminal case: "and he said hurry up comma fucking hurry up stop end quote." At the end of sitting weeks in Canberra, when everyone would rush home, Kirby would rush back to his Sydney chambers. One day, as we walked across the tarmac to the plane, he tipped his face upwards and said, squinting, "This is my five minutes of sunlight." The hiatus from work was brief. Seated in the nose of the Dash 8, as the plane turned east over the Blue Mountains, Kirby took out a pen and set about revising a speech.

Among the mementoes in his Canberra chambers, Kirby displayed a photo of himself with John Howard. The two men stand at a function, almost - but not quite - shoulder to shoulder. That the Kirby years were the Howard years (Kirby was appointed to the High Court in 1996) is easy to recite. It's harder to quantify how the times shaped Kirby's position within the court, and his own sense of duty in light of it. The photo gives nothing away. This is, as Inga Clendinnen has noted, the problem with photographs - and with those "hidden movements of the imagination" that are the meanings we ascribe to them. "We know what we as individuals feel, but we cannot know whether we are idiosyncratic. To investigate that, we have to formulate thoughts into words, and then offer them for discussion."

Kirby's efforts to formulate his thoughts and offer them for discussion, everywhere in his judgements and letters, are also in the tales that every second person (or so it sometimes seems) has to tell about him: a speech heard at a university or an event, or a dinner attended, and remembered ever since. From these, Kirby emerges as both the rhetorician, persuading, and the philosopher, refuting; but all the while willing to articulate and test his assumptions, to be frank about his views and, sometimes, to change them.

In May 2007, Kirby performed a hip-hop duet at the launch of Arts Law Week in Melbourne. Calling for respect for human rights, he recited Yeats's poem ‘He Wishes for the Cloths of Heaven' while the hip-hop artist Elf Tranzporter beat-boxed in the background. Kirby was photographed removing his suit jacket to reveal a bright yellow blazer as he announced to the packed audience that he was preparing for life after the bench.

On the day that the Howard government finally announced it would, if re-elected, give equal superannuation rights to same-sex partners, a spam artist sent Kirby an unrelated, garbled email with the enticing subject line "EVERYTHING IS POSSIBLE". Ross Abbs, Kirby's Sydney-based associate at the time, printed the email and left it in the judge's in-tray. It came back with a scrawled note: "Represent when I fall off the perch." At first Ross thought Kirby was using the word ‘represent' the way rappers use it: to stand up for, to be a role model, to give respect to. Represent when I fall off the perch. Then Ross realised Kirby meant ‘re-present'; he wished to be shown the email again when he retired, to be reminded that everything is possible.

Not long afterwards, I finished working at the court. Late on a Sunday night, as I cleared out my desk in the Canberra chambers, Kirby came in from his flight from Sydney, ready for the week of sittings. He helped me to pick up my things. With my arms full and a pen between my teeth, I slammed the door shut behind us. "A most inelegant exit," said the judge. "With something clamped in your mouth like a chimpanzee. The end." We crossed the dark floor to the lifts. "It won't be long before I'll be leaving, too."

As I drove away, I remembered a dinner the week before. Kirby had spoken of his imminent departure, and the future. "You have to look at life as if it's a grand nineteenth-century novel." The wall of the Thai restaurant was a brilliant marigold behind him. "A Joseph Conrad tale. This highway robbery, that love affair, now this time of servitude. And so on."