Prune parameters in a given layer

[Sheethal00]

Hi,

I am trying remove some filters in a Conv2D layer based on their l2 norm values.

Ex:
If a Conv2D layer has 32 filters with size (3,3,3,32) 3x3 is size of the kernel, 3 is repeating gray scale image and 32 is number of filters

Can i remove some filters and make the layer shape (3,3,3,30) with your codebase

I dint really go through your codebase

Thanks

[Arexh]

Hi @Sheethal00 , that probably won't work. The method I use can only be applied to prune on the fully connected layer. Since BERT has a lot of params in attention layers which are fully connected layers, this method works on the BERT model. If you are interested, you can read the paper 1910.04732 :)