The novel is dead (this time it's for real)

If you happen to be a writer, one of the great benisons of having children is that your personal culture-mine is equipped with its own canaries. As you tunnel on relentlessly into the future, these little harbingers either choke on the noxious gases released by the extraction of decadence, or they thrive in the clean air of what we might call progress. A few months ago, one of my canaries, who's in his mid-teens and harbours a laudable ambition to be the world's greatest ever rock musician, was messing about on his electric guitar. Breaking off from a particularly jagged and angry riff, he launched into an equally jagged diatribe, the gist of which was already familiar to me: everything in popular music had been done before, and usually those who'd done it first had done it best. Besides, the instant availability of almost everything that had ever been done stifled his creativity, and made him feel it was all hopeless.

A miner, if he has any sense, treats his canary well, so I began gently remonstrating with him. Yes, I said, it's true that the web and the internet have created a permanent Now, eliminating our sense of musical eras; it's also the case that the queered demographics of our longer-living, lower-birthing population means that the middle-aged squat on top of the pyramid of endeavour, crushing the young with our nostalgic tastes. What's more, the decimation of the revenue streams once generated by analogues of recorded music have put paid to many a musician's income. But my canary had to appreciate this: if you took the long view, the advent of the 78rpm shellac disc had also been a disaster for musicians who in the teens and 20s of the last century made their daily bread by live performance. I repeated one of my favourite anecdotes: when the first wax cylinder recording of Feodor Chaliapin singing "The Song of the Volga Boatmen" was played, its listeners, despite a lowness of fidelity that would seem laughable to us (imagine a man holding forth from a giant bowl of snapping, crackling and popping Rice Krispies), were nonetheless convinced the portly Russian must be in the room, and searched behind drapes and underneath chaise longues for him.

So recorded sound blew away the nimbus of authenticity surrounding live performers – but it did worse things. My canaries have often heard me tell how back in the 1970s heyday of the pop charts, all you needed was a writing credit on some loathsome chirpy-chirpy-cheep-cheeping ditty in order to spend the rest of your born days lying by a guitar-shaped pool in the Hollywood Hills hoovering up cocaine. Surely if there's one thing we have to be grateful for it's that the web has put paid to such an egregious financial multiplier being applied to raw talentlessness. Put paid to it, and also returned musicians to the domain of live performance and, arguably, reinvigorated musicianship in the process. Anyway, I was saying all of this to my canary when I was suddenly overtaken by a great wave of noxiousness only I could smell. I faltered, I fell silent, then I said: sod you and your creative anxieties, what about me? How do you think it feels to have dedicated your entire adult life to an art form only to see the bloody thing dying before your eyes?

My canary is a perceptive songbird – he immediately ceased his own cheeping, except to chirrup: I see what you mean. The literary novel as an art work and a narrative art form central to our culture is indeed dying before our eyes. Let me refine my terms: I do not mean narrative prose fiction tout court is dying – the kidult boywizardsroman and the soft sadomasochistic porn fantasy are clearly in rude good health. And nor do I mean that serious novels will either cease to be written or read. But what is already no longer the case is the situation that obtained when I was a young man. In the early 1980s, and I would argue throughout the second half of the last century, the literary novel was perceived to be the prince of art forms, the cultural capstone and the apogee of creative endeavour. The capability words have when arranged sequentially to both mimic the free flow of human thought and investigate the physical expressions and interactions of thinking subjects; the way they may be shaped into a believable simulacrum of either the commonsensical world, or any number of invented ones; and the capability of the extended prose form itself, which, unlike any other art form, is able to enact self-analysis, to describe other aesthetic modes and even mimic them. All this led to a general acknowledgment: the novel was the true Wagnerian Gesamtkunstwerk.

This is not to say that everyone walked the streets with their head buried in Ulysses or To the Lighthouse, or that popular culture in all its forms didn't hold sway over the psyches and imaginations of the great majority. Nor do I mean to suggest that in our culture perennial John Bull-headed philistinism wasn't alive and snorting: "I don't know much about art, but I know what I like." However, what didn't obtain is the current dispensation, wherein those who reject the high arts feel not merely entitled to their opinion, but wholly justified in it. It goes further: the hallmark of our contemporary culture is an active resistance to difficulty in all its aesthetic manifestations, accompanied by a sense of grievance that conflates it with political elitism. Indeed, it's arguable that tilting at this papery windmill of artistic superiority actively prevents a great many people from confronting the very real economic inequality and political disenfranchisement they're subject to, exactly as being compelled to chant the mantra "choice" drowns out the harsh background Muzak telling them they have none.

Just because you're paranoid it doesn't mean they aren't out to get you. Simply because you've remarked a number of times on the concealed fox gnawing its way into your vitals, it doesn't mean it hasn't at this moment swallowed your gall bladder. Ours is an age in which omnipresent threats of imminent extinction are also part of the background noise – nuclear annihilation, terrorism, climate change. So we can be blinkered when it comes to tectonic cultural shifts. The omnipresent and deadly threat to the novel has been imminent now for a long time – getting on, I would say, for a century – and so it's become part of culture. During that century, more books of all kinds have been printed and read by far than in the entire preceding half millennium since the invention of movable-type printing. If this was death it had a weird, pullulating way of expressing itself. The saying is that there are no second acts in American lives; the novel, I think, has led a very American sort of life: swaggering, confident, brash even – and ever aware of its world-conquering manifest destiny. But unlike Ernest Hemingway or F Scott Fitzgerald, the novel has also had a second life. The form should have been laid to rest at about the time of Finnegans Wake, but in fact it has continued to stalk the corridors of our minds for a further three-quarters of a century. Many fine novels have been written during this period, but I would contend that these were, taking the long view, zombie novels, instances of an undead art form that yet wouldn't lie down.

Literary critics – themselves a dying breed, a cause for considerable schadenfreude on the part of novelists – make all sorts of mistakes, but some of the most egregious ones result from an inability to think outside of the papery prison within which they conduct their lives' work. They consider the codex. They are – in Marshall McLuhan's memorable phrase – the possessors of Gutenberg minds.

There is now an almost ceaseless murmuring about the future of narrative prose. Most of it is at once Panglossian and melioristic: yes, experts assert, there's no disputing the impact of digitised text on the whole culture of the codex; fewer paper books are being sold, newspapers fold, bookshops continue to close, libraries as well. But … but, well, there's still no substitute for the experience of close reading as we've come to understand and appreciate it – the capacity to imagine entire worlds from parsing a few lines of text; the ability to achieve deep and meditative levels of absorption in others' psyches. This circling of the wagons comes with a number of public-spirited campaigns: children are given free books; book bags are distributed with slogans on them urging readers to put books in them; books are hymned for their physical attributes – their heft, their appearance, their smell – as if they were the bodily correlates of all those Gutenberg minds, which, of  course, they are.

The seeming realists among the Gutenbergers say such things as: well, clearly, books are going to become a minority technology, but the beau livre will survive. The populist Gutenbergers prate on about how digital texts linked to social media will allow readers to take part in a public conversation. What none of the Gutenbergers are able to countenance, because it is quite literally – for once the intensifier is justified – out of their minds, is that the advent of digital media is not simply destructive of the codex, but of the Gutenberg mind itself. There is one question alone that you must ask yourself in order to establish whether the serious novel will still retain cultural primacy and centrality in another 20 years. This is the question: if you accept that by then the vast majority of text will be read in digital form on devices linked to the web, do you also believe that those readers will voluntarily choose to disable that connectivity? If your answer to this is no, then the death of the novel is sealed out of your own mouth.

We don't know when the form of reading that supported the rise of the novel form began, but there were certain obvious and important way-stations. We think of Augustine of Hippo coming upon Bishop Ambrose in his study and being amazed to see the prelate reading silently while moving his lips. We can cite the introduction of word spaces in seventh-century Ireland, and punctuation throughout medieval Europe – then comes standardised spelling with the arrival of printing, and finally the education reforms of the early 1900s, which meant the British Expeditionary Force of 1914 was probably the first universally literate army to take to the field. Just one of the ironies that danced macabre attendance on this most awful of conflicts was that the conditions necessary for the toppling of solitary and silent reading as the most powerful and important medium were already waiting in the wings while Sassoon, Graves and Rosenberg dipped their pens in their dugouts.

In Understanding Media, Marshall McLuhan writes about what he terms the "unified electrical field". This manifestation of technology allows people to "hold" and "release" information at a distance; it provides for the instantaneous two‑way transmission of data; and it radically transforms the relationship between producers and consumers – or, if you prefer, writers and readers. If you read McLuhan without knowing he was writing in the late 1950s, you could be forgiven for assuming he was describing the interrelated phenomena of the web and the internet that are currently revolutionising human communications. When he characterises "the global village" as an omni-located community where vast distances pose no barrier to the sharing of intimate trivia, it is hard not to believe he himself regularly tweeted. In fact, McLuhan saw the electric light and the telegraph as the founding technologies of the "unified electrical field", and, rather than being uncommonly prescient, he believed all the media necessary for its constitution – broadcast radio, film, television, the telephone – were securely in place by the time of, say, the publication of Finnegans Wake.

McLuhan, having enjoyed his regulation 15 minutes of fame in the unified electrical field of the 1960s has fallen out of fashion; his rigorous insistence that the content of any given medium is an irrelevance when it comes to understanding its psychological impact is unpopular with the very people who first took him up: cultural workers. No one likes to be told their play/novel/poem/film/TV programme/concept double-album is wholly analysable in terms of its means of transmission. Understanding Media tells us little about what media necessarily will arise, only what impact on the collective psyche they must have. In the late 20th century, a culture typified by a consumerist ethic was convinced that it – that we – could have it all. This "having it all" was even ascribed its own cultural era: the postmodern. We weren't overtaken by new technologies, we simply took what we wanted from them and collaged these fragments together, using the styles and modes of the past as a framework of ironic distancing: hence the primacy of the message was reasserted over its medium.

The main objection to this is, I think, at once profoundly commonsensical and curiously subtle. The literary critic Robert Adams observed that if postmodernism was to be regarded as a genuine cultural era, then it made modernism itself a strangely abbreviated one. After all, if we consider that all other western cultural eras – classicism, medieval, the Renaissance – seem to average about half a millennium a piece, it hardly matters whether you date modernism's onset to Rousseau, Sturm und Drang or Les Demoiselles d'Avignon, it clearly still has a long way to go. By the same token, if – as many seem keen to assert – postmodernism has already run its course, then what should we say has replaced it, post-postmodernism, perhaps? It would seem better all round to accept the truth, which is that we are still solidly within the modernist era, and that the crisis registered in the novel form in the early 1900s by the inception of new and more powerful media technologies continues apace. The use of montage for transition; the telescoping of fictional characters into their streams of consciousness; the abandonment of the omniscient narrator; the inability to suspend disbelief in the artificialities of plot – these were always latent in the problematic of the novel form, but in the early 20th century, under pressure from other, juvenescent, narrative forms, the novel began to founder. The polymorphous multilingual perversities of the later Joyce, and the extreme existential asperities of his fellow exile, Beckett, are both registered as authentic responses to the taedium vitae of the form, and so accorded tremendous, guarded respect – if not affection.

After Joyce, we continue to read; we read a great deal – after all, that's what you do when you're wheeled out into the sun porch of a care home: you read. You may find it difficult to concentrate, given the vagaries of your own ageing Gutenberg mind, while your reading material itself may also have a senescent feel, what with its greying stock and bleeding type – the equivalent, in codex form, of old copies of the Reader's Digest left lying around in dentists' waiting rooms. Yet read you do, closing your ears obstinately to the nattering of radio and television, squinting so as to shut out the bluey light from the screens that surround you, turning your head in order to block out the agitation of your neighbours' fingers as they tweezer info panels into being. I've often thought that western European socialism survived as a credible ideological alternative up until 1989 purely because of the Soviet counterexample: those on the left were able to point east and say, I may not altogether know how socialism can be achieved, but I do know it's not like this. So it was with the novel: we may not have known altogether how to make it novel again, but we knew it couldn't go the way of Hollywood. Now film, too, is losing its narrative hegemony, and so the novel – the cultural Greece to its world-girdling Rome – is also in ineluctable decline.

I repeat: just because you're paranoid it doesn't mean they aren't out to get you. When I finished my first work of fiction in 1990 and went looking for a publisher, I was offered an advance of £1,700 for a paperback original edition. I was affronted, not so much by the money (although pro rata it meant I was being paid considerably less than I would have working in McDonald's), but by not receiving the sanctification of hard covers. The agent I consulted told me to accept without demur: it was, he said, nigh-on impossible for new writers to get published – let alone paid. At that time the reconfiguration of the medium was being felt through the ending of the Net Book Agreement, the one-time price cartel that shored up publishers' profits by outlawing retailer discounting. In retrospect, the ending of the agreement was simply a localised example of a much wider phenomenon: the concertinaing of the textual distribution network into a short, wide pipe. It would be amusing to read the meliorism of the Panglosses if it weren't also so irritating; writing a few months ago in the New Statesman, Nicholas Clee, a former editor of the Bookseller, no less, surveyed all of the changes wrought by digital media – changes that funnel together into the tumultuous wordstream of Jeff Bezos's Amazon – before ending his excursus where he began, with the best of all possible facts implying we were in the best of all possible worlds: "I like," Clee wrote, "buying books on Amazon."

Groucho Marx once said to a man with six children taking part in his TV show: "I like my cigar, but I know when to take it out." By the same token: I also like buying books on Amazon, but I'm under no illusion that this means either the physical codex, or the novel – a form of content specifically adapted to it – will survive as a result of my preferences. Because I'm also very partial to sourcing digital texts from Project Gutenberg, then wordsearching them for a quotation I want to use. I like my typewriter as well, a Groma Kolibri manufactured in the German Democratic Republic in the early 1960s, but I'm under no illusion that it's anything but old technology. I switched to writing the first drafts of my fictions on a manual typewriter about a decade ago because of the inception of broadband internet. Even before this, the impulse to check email, buy something you didn't need, or goggle at images of the unattainable was there – but at least there was the annoying tocsin of dial-up connection to awake you to your time-wasting. With broadband it became seamless: one second you were struggling over a sentence, the next you were buying oven gloves. Worse, if, as a writer, you reached an impasse where you couldn't imagine what something looked or sounded like, the web was there to provide instant literalism: the work of the imagination, which needs must be fanciful, was at a few keystrokes reduced to factualism. All the opinions and conceptions of the new media amount to nothing set beside the way they're actually used.

While I may have registered the effect of digital media on my sense perception, I by no means feel immune from them; on the contrary, I've come to realise that the kind of psyche implicit in the production and consumption of serious novels (which are what, after all, serious artists produce), depends on a medium that has inbuilt privacy: we must all be Ambroses. In a recent and rather less optimistic article in the New Yorker on the Amazon phenomenon, George Packer acknowledges the impact on the publishing industry of digital text: the decline in physical sales; and the removal of what might be termed the "gatekeepers", the editors and critics who sifted the great ocean of literary content for works of value. He foresees a more polarised world emerging: with big bestsellers commanding still more sales, while down below the digital ocean seethes with instantly accessible and almost free texts. Packer observes that this development parallels others in the neoliberal economy, which sees market choice as the only human desideratum. The US court's ruling against the big five publishers in the English-speaking world and in favour of Amazon was predicated on this: their desperate attempt to resist Amazon's imposition of punitive discounting constituted a price cartel. But, really, this was only the latest skirmish in a long war; the battles of the 1990s, when both here and in the US chain bookstores began to gobble up the independents, were part of the same conflict: one between the medium and the message, and as I think I've already made clear, in the long run it's always the medium that wins.

I've no doubt that a revenue stream for digitised factual text will be established: information in this form is simply too useful for it not to be assigned monetary value. It is novels that will be the victims of the loss of effective copyright (a system of licensing and revenue collection that depended both on the objective form of the text, and defined national legal jurisdictions); novels and the people who write them. Fortunately, institutions are already in existence to look after us. The creative writing programmes burgeoning throughout our universities are exactly this; another way of looking at them is that they're a self-perpetuating and self-financing literary set-aside scheme purpose built to accommodate writers who can no longer make a living from their work. In these care homes, erstwhile novelists induct still more and younger writers into their own reflexive career paths, so that in time they too can become novelists who cannot make a living from their work and so become teachers of creative writing.

In case you think I'm exaggerating, I have just supervised a doctoral thesis in creative writing: this consists in the submission of a novel written by the candidate, together with a 35,000-word dissertation on the themes explored by that novel. My student, although having published several other genre works, and despite a number of ringing endorsements from his eminent creative-writing teachers, has been unable to find a publisher for this, his first serious novel. The novel isn't bad – although nor is it Turgenev. The dissertation is interesting – although it isn't a piece of original scholarship. Neither of them will, in all likelihood, ever be read again after he has been examined. The student wished to bring the date of his viva forward – why? Well, so he could use his qualification to apply for a post teaching – you guessed it – creative writing. Not that he's a neophyte: he already teaches creative writing, he just wants to be paid more highly for the midwifery of stillborn novels.

If you'll forgive a metaphoric ouroboros: it shouldn't surprise us that this is the convulsive form taken by the literary novel during its senescence; some of the same factors implicated in its extinction are also responsible for the rise of the creative writing programme; specifically a wider culture whose political economy prizes exchange value over use value, and which valorises group consciousness at the expense of the individual mind. Whenever tyro novelists ask me for career advice I always say the same thing to them: think hard about whether you wish to spend anything up to 20 or 30 years of your adult life in solitary confinement; if you don't like the sound of that silence, abandon the idea right away. But nowadays many people who sign up for creative-writing programmes have only the dimmest understanding of what's actually involved in the writing life; the programme offers them comity and sympathetic readers for their fledgling efforts – it acts, it essence, as a therapy group for the creatively misunderstood. What these people are aware of – although again, usually only hazily – is that some writers have indeed had it all; if by this is meant that they are able to create as they see fit, and make a living from what they produce. In a society where almost everyone is subject to the appropriation of their time, and a vast majority of that time is spent undertaking work that has little human or spiritual value, the ideal form of the writing life appears gilded with a sort of wonderment. The savage irony is that even as these aspirants sign up for the promise of such a golden career, so the possibility of their actually pursuing it steadily diminishes; a still more savage irony is that the very form their instruction takes militates against the culture of the texts they desire to produce. WB Yeats attributed to his father the remark that "Poetry is the social act of the solitary man"; with the creative-writing programmes and the Facebook links embedded in digitised texts encouraging readers to "share" their insights, writing and reading have become the solitary acts of social beings. And we all know how social beings tend to regard solitary acts – as perversities, if not outright perversions.

As I said at the outset: I believe the serious novel will continue to be written and read, but it will be an art form on a par with easel painting or classical music: confined to a defined social and demographic group, requiring a degree of subsidy, a subject for historical scholarship rather than public discourse. The current resistance of a lot of the literate public to difficulty in the form is only a subconscious response to having a moribund message pushed at them. As a practising novelist, do I feel depressed about this? No, not particularly, except on those occasions when I breathe in too deeply and choke on my own decadence. I've no intention of writing fictions in the form of tweets or text messages – nor do I see my future in computer-games design. My apprenticeship as a novelist has lasted a long time now, and I still cherish hopes of eventually qualifying. Besides, as the possessor of a Gutenberg mind, it is quite impossible for me to foretell what the new dominant narrative art form will be – if, that is, there is to be one at all.

What I can do is observe my canary: he doesn't read much in the way of what I'd call serious novels, but there's no doubting that he's alive, breathing deep of a rich and varied culture, and shows every sign of being a very intelligent and thoughtful songbird. On that basis, I think it's safe for us both to go on mining.