Hey Carl,
Totally agree, adding data nilly-willy will most of the time make the needle in the haystack harder to find. This is why I don’t call it a ‘trick’ but instead a ‘strategy’. Adding data will always give the model more context, but it’s about finding which data will help clarify the model’s view of the data and which ones may cloud it. Consider adding more data in the sense I am talking about — intelligently adding data based on real-world context and knowledge of how the model operates — as building a better needle-finding device instead of adding more hay.
Cheers!