Hello Akshay! Glad you enjoyed.

The network chooses a subnetwork based on random initialization, when weights are in convenient places. If we were to take those subnetworks out and put them back in, presumably the network would not explore other networks because the subnetwork weights have already been trained (probably still 5% of the neurons would be used if the subnetwork is 5% of the network).

Perhaps I didn’t understand your comment correctly — let me know if I made a mistake.

Cheers!

ML & CS enthusiast. Let’s connect: https://www.linkedin.com/in/andre-ye.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store