One of those who took issue with me about this from time to time was an insurance salesman I used to bump into quite often because we tended to use the same pubs in the town.
We enjoyed many an evening of political banter, so I was quite surprised when I found him in partnership with the then local Liberal councillor Colin Jones in raising much-needed cash for vital health equipment which was either being axed or not provided for Haverhill (no change there then).
He masterminded an amazing campaign which saved one of the town's ambulances from being axed, and which provided defibrillators and numerous other life-saving devices for them.
The campaign was called HaverHeart and the man who organised it was Ted Trebble.
He moved away, but found, like many others, that Haverhill held a special place in his affections, so much so that he moved back, a fact which he announced to me by letter some months before it happened, as if it were a great achievement.
He became an occasional correspondent with this newspaper, always with something interesting and sensible to say.
But I never thought he would go into politics himself.
Retirement gave him the time, and the debacle of the HRA gave him the opportunity to be involved in a revival of Tory fortunes in the town.
In this way it gave him a platform to express the most central tenet of his views about Haverhill – that the town had never had a fair deal from the borough council in Bury St Edmunds.
Many have held that belief but few have had the understanding, ability and sheer guts to do anything about it.
But from the HaverHeart years I learned Ted had those qualities and, although he started cautiously like a good Test batsman, he was lining up for some spectacular shots and a big hundred until he got bowled a googly from an unexpected direction.
It cut him short in his prime and robbed Haverhill of this momentous opportunity for a local politician with the right qualities who had Haverhill totally at heart, would work with all parties, was uninterested in political posturing and not put off by attempted bullying or bamboozling from local government or statutary bodies.
I interviewed him at length when he became mayor and he outlined several ideas he was working on – new and realistic things, not just responding to others, playing politics or repeating the same old vague impossible hopes and aspirations.
A week later the man who raised so much for life-saving equipment – saying 'you never know when you're going to need it yourself' – was himself struck down and in need of care.
He retained his interest in the town to the end, keeping in close touch with the town council and even ringing me from his hospital bed to tell me what he was planning when he was back on his feet.
Sadly for him, and for Haverhill, that wasn't to be.