The BBC’s Listening Project turns seven this month. It offers a window into a gentler, more compassionate Britain

What was it? At first I couldn’t tell. I was listening to the radio on my headphones, and a woman was speaking, in a northern accent, gently, into my right ear, about her mother having died when she was young, and how after that she had wanted her father near her all the time. Suddenly I felt there was someone else nearby, who was not the speaker, so I removed the headphones to say hello. But there wasn’t anyone there. I replaced them again, and listened to Cynthia say she used to write notes for her father to find when he came home from the pit, about how she needed to speak to him, urgently.

Again, that visceral sense of a presence; again, no one there. Headphones back on – of course, there was nothing urgent, “I just wanted to look at him.” “I can remember getting me dad to come and lie next to me on the bed before he went to work,” said a voice full of tears, into my other ear, “I used to think, if I hold his hand really tight, he won’t be able to go to work” – and the mystery was solved. It was her sister’s listening presence I had felt, so strong it was almost in the room with me.

The BBC’s The Listening Project – in which ordinary Britons talk to each other about important moments in their lives, moments that, however well they know each other, they have not really discussed before – was inspired by an American project, StoryCorps, which houses the results in the Library of Congress. The Listening Project conversations, broadcast at about three minutes, are archived at full length in the British Library, which thereby builds an aural snapshot of experience and feeling that this month is seven years old and many lifetimes deep. The range of subjects is huge, from the seemingly anecdotal (a second world war bomb depositing a functioning sewing machine on someone’s windowsill), to the foundational (lost children, lost parents); from a nude model telling a nervous photographer how he was the first to make her feel seen as herself; to a gay son and his gay mother comparing coming-out stories, to an ex-oil worker telling his young daughter about his survivor’s guilt, because he was ashore when his colleagues died on the Piper Alpha oil platform in 1988. She remembers cycling to the memorial service – “that was really fun” – but also the little box for the ashes of a man who couldn’t be identified.

It’s an extraordinary thing, a privilege to have. What you also realise, however, hearing the phatic sounds that are basically polite place-holders until it’s someone’s turn to speak; the pile-up of anecdotes that are not so much responses as syncopated lists, is just how hard it is to really listen. To find the humility to set aside the needs of self and listen to what is said, as opposed to what we supply, or what, for whatever reason – comfort, ammunition, self-ratification – we need to hear. And you realise how often, elsewhere – in the performance of interruption that is Radio 4’s Today programme, for instance, or the blinkered aggressions of the House of Commons, or the curated self-presentations of social media – proper listening isn’t happening at all. The shadow chancellor John McDonnell’s recent promise of a “mammoth, massive listening exercise” after MPs had quit his party only underlines the lack.

One of the moving things about the conversations in The Listening Project is how many of the participants are trying to get past these hurdles. Not all succeed, of course. And perhaps it is especially difficult for those who are talking to family: so much baggage accrues, so many assumptions, often laid down so long ago they have taken on the heft of truth. Which makes the moments when they can begin to be cleared away all the stranger, and more surprising. Maggie tells Cynthia how much she admired her, how much she cared. “Oh, Maggie,” answers Cynthia, who has clearly seen her sister a lot over the last few decades, but is taken aback nonetheless, “I can’t believe anybody looked up to me.”

The same applies to the larger family: the nation. So much of the national conversation at the moment is based on what people want to remember, on what was apparently a better time. But listen to the detail, from those who were there, and yes, the passports were blue, but also there was peacetime conscription, and parents, uncaring or very much caring, had to send their children away and not know if they would ever see them again.

So we get Phyliss and Freda, both evacuees, Phyliss from an orphanage, Freda from her family (Freda “can’t remember [my mother] cuddling us, I didn’t miss that … I never had it”) but both nonetheless wondering at how children were herded into rooms to be picked out by strangers. “They wouldn’t do that now, would they?”

Or friends Ruth and Ruth remember leaving Germany on the Kindertransport, one of the Ruths still unable to come to terms with how her mother could “sacrifice the love of a daughter”, but not her two brothers. Nor is she able to come to terms with the stone-throwing she met here, for being Jewish, and foreign, and German; she has always felt the antisemitism so strongly that until recently she often didn’t admit to being Jewish.

Last year I published a book based on many hours of listening to my grandmother. At the various events I subsequently held, in rural Scotland, or Wales, or London or Addis Ababa, I was struck by the number of people – English, American, Scottish, Ethiopian of course – who said to me, “But that’s my grandmother.” Or, “That’s exactly how I’ve felt.” Or, “I wish I’d listened to my own grandparents before they passed away.”

My grandmother grew up in northern Ethiopia, in a feudal empire. Utterly different to most of those people’s backgrounds, you’d think – but it turns out that the funny thing about listening hard to the idiosyncratic particulars, to people talking about themselves, in their own words, about what happened to them and how they felt about it, is the degree to which the universals emerge: the things we all share. And in this short-termist, alarmist, future-terrified time, that seems to me so important. “You were the sweetest little thing,” says Cynthia to Maggie. “You were born in the room next door. And I heard your first cry. I cried with you.”

• Aida Edemariam is a journalist and author. Her latest book is The Wife’s Tale