It really depends what you play vs them.
Destro rely on getting off several chaos bolts after one-another, which with los-ing, interrupts and stun is quite easy in most cases.
What you need to be aware of is Unending resolve which makes them immune to interrupts for 8 sec and nether ward which reflects magic on the warlock for 5 seconds(counterspell, wind shear etc).
Angrypie mentioned shadowburn spec, which is more viable in 3s than in 2s, but I still use it vs ferals/rogues who can interrupt CB spec pretty badly with LOS-ing and interrupts.
Destro actually has pretty low survival now compared to earlier. You have to make them pop unending resolve and dark pact, both on a 1min cd, then they have no real heals.
If you see destro use unending resolve offensively to get of a few Chaos Bolts, then you probably can kill him after, granted you can rotate some cc on healer.
Havoc isn't really used for hitting both players with chaos bolts anymore, due to "focused chaos", the talent which increases chaos bolt damage. It's more for applying immolate, conflag roots and curse.
If you find a warlock with long duration havoc on, then he's probably using the mentioned shadowburn spec, which is pretty easy to outheal for most healers.