Nick Cave is the greatest rock artist Australia has produced. No contest. Somewhere in history there may have been a contender who never got his or her chance. But with Cave, album after album, shots go off, every couple of years bringing a new bolt of intensity: great songs, good songs, a few bad songs, packed, positioned and aimed at an ever-growing fan-base and a critical establishment baying for his blood. His blood to drink. He’s the man.

Much is made of the changes in the man, from skinny St Kilda insect to world cultural force. From the outside he looks more professional than he was, but in the tender of his craft he has always been diligent, bloody-minded. Even in the early days, amid the drugs and post-punk fashion, his work ethic was just as strong, the whole game being about artistic integrity and a maniacal striving to express himself. Now the worlds of ballet, opera, theatre and film fall over him. Fashion designers pay homage to his look. He writes books. He curates festivals, festivals that buckle to his will and schedule. Still he conducts himself just the same, with fantastic grace, nothing tacky. In interviews he is thoughtful, polite, humorous. And all this in an industry built on distortion and slash and burn, on the quick buck and the capacity to eat its own young.

The Bad Seeds, once whispered to be Nick Cave and The Cavemen, are the demented beat group who will follow him anywhere. Handpicked over three continents, like a jewel-heist movie collection of individuals, each is the top man in his own craft. It’s the leader who holds them together, torch in hand. He’s after world domination but he’d settle, just as easy, for a good show every night.

The new album, B-sides and Rarities, arrives hot on the heels of late last year’s Abbatoir Blues/The Lyre of Orpheus, which suggests that Cave and his band plan to keep up their frenetic recording schedule. Normally an album like this would be held up a year until the last record has run its course. It’s a three-CD set and it comes, as did the last one, in a box, a black box with silver writing. There are no band photos, no glossy booklet; it’s not chock-a-block with the fan bric-a-brac these types of releases usually engender. Instead it’s all very Bad Seeds, muted and stark, with just the plain facts laid out in a classy, artful package.

B-sides are interesting things. In the hands of a great band or artist they can be full of rich pickings. Mostly they come from two sources: either they are songs recorded at an album session but deemed not “right” or good enough for the album, or they are recorded after the album when the record company is hunting for something to accompany the singles it plans to release. Both situations enable fantastic songs to get through. Furthermore bands, in the heat of the studio, with deadlines to meet, confronted with the enormousness of recording an album, are notoriously bad at choosing which songs to put on it. Good songs get left off and land on B-sides. Also, the time after an album can be fertile for a writer and, in the month or two after recording, a new song can pop up that’s good, or really good, and the writer gets excited, with a studio booked and the single ready to go. He records it and then it’s gone, waiting for compilations like these.

So what do we have here? Fifty-six songs, spread over three volumes, offer a rough alternative history of the band. Certain moments suddenly stick out as turning points. The first tinkling piano notes of “The Train Song” (the B-side to “The Ship Song”) herald Cave’s arrival as a serious balladeer. This is the sound we will eventually get to know, and here it is, at its inception, in 1990.

And what comes immediately before it? Two numbers, two amazing songs, which now, in this sequencing, sound like the last two great screams of the band as they were. “Scum” and “The Girl at the Bottom of My Glass” are rough and guitar-based, stripped down and mesmerising, buzzing. The latter is simply Cave and Mick Harvey on guitars. Later they team up for a fierce acoustic version of “Jack The Ripper”, from 1992, that shreds the album version.

The arrival of Cave as piano player changes everything. Gone is the ramshackle, bluesy, six-strings-that-drew-blood band. Those tinkling notes come in, the line-up settles. And everything is anew and as we know it. The ballads roll out. “Blue Bird”, “Cassiel’s Song” and “What Can I Give You?” are all here, and all good, although the better ones are on the albums, Henry’s Dream and Let Love In.

The next big jolt is the Murder Ballads period, a musical form not up my street and a record I skipped. A fallow stage in the band’s history follows and it depends where you fall with this, whether you believe the albums are completely successful or not. Here, though, “The Ballad of Robert Moore and Betty Coltrane” (the B-side to “Where the Wild Roses Grow”) kicks out nicely from the ballads. So does the 18-minute “O’Malley’s Bar (Parts 1, 2, 3 & Reprise)”, culled from a 1996 radio show. Then come two classics. Funny how great albums give good B-sides; from The Boatman’s Call out-takes we get “Little Empty Boat” and “Right Now I’m A-Roaming”. Both strike a pitch of humour and malice, of tight muscular musical control and killer lyrics. For all the praise that rains on the violence of Cave, he’s a funny man, and when he gets a nice little melody – usually a folk tune – and decides to get silly he’s terrific.

The next two albums, No More Shall We Part and Nocturama, find the band in a vice, kicking, and the B-sides tell the story. They are mostly slow, with lots of piano and organ, beautifully sung and arranged but a little lifeless, and taken as a pack, as they are laid out here, it can be heavy going. The wonderful wake-up is “She’s Leaving You”, the flip-side to Abattoir Blues’s “Nature Boy”. Abattoir Blues/The Lyre of Orpheus is an album I really didn’t think they had in them. If the record is about one thing it’s liberation, and it’s bursting out everywhere. Cave’s singing is passionate, he’s got a brace of great rock songs and the band is exploding all over the place. There’s a feeling of “we can do anything”. “She’s Leaving You” is a magnificent comic pile-up, driven by a Bad Seeds who have never sounded better. It’s a happy ending.

Nick Cave has described this new collection as his favourite album, which is a typical piece of Nick Cave perversity, and yet you can see why he would. It’s like him. It’s big: three CDs and 219 minutes. And it swings and wails like he punches. It has epics, jokes, blues, hard heartbreaking ballads and lots of very sad songs with the words “sea”, “snow”, “sorrow” and “babe” repeated over and over. Which is not to knock him; simply to acknowledge that even on his most consistent and levelled recordings, he is uncontainable. This widescreen album, filled as it is with the great, the good and the failed, fits him, suits him, more than any album ever has.