I must admit I'm also often tempted to take my FX 15-20 over the limit (worded not to admit anything!) because it's so easy and so non-disruptive and fun.
It should be possible to anticipate most incidents of this type and be accommodating to other road users, bearing in mind that some granny or distracted mother or lost driver isn't going to have very good reactions compared to yours. It's their responsibility to look, but you're like a silent rocket appearing out of nowhere and nobody is perfect.
You have good focus, but also note that the most horrible of accidents are usually the result of multiple factors coinciding. So while you easily avoid a narrow miss, it is when something else unexpected happens during the time your attention is occupied that you get T-boned and such. Also, when a car is behaving erratically, hold back to see what foolish move they are about make (sudden U-turn or left turn is common).
So if you're getting more than one close call a year, you should chill (just a little) because of statistics, partly because you can't change the whole world, and partly because it's better to be alive than correct.