Answer by greg for Rule chain matrix by vector

Take an ordinary scalar function $\phi(z)$ and its derivative$\phi'(z)=\frac{d\phi}{dz}$ and apply them element-wise to a vector argument, i.e. $$\eqalign{v &= X\beta,\quadf &= \phi(v),\quadf'&= \phi'(v) \cr}$$The differential of such a vector function can be expressed using an elementwise $(\odot)$ product or better yet, a Diagonal matrix $$\eqalign{df &= f'\odot dv \cr&= {\rm Diag}(f')\,dv \cr&= {\rm Diag}(f')\,X\,d\beta \cr}$$Given this differential, the gradient with respect to $\beta$ can be identified as the matrix
$$\eqalign{\frac{\partial f}{\partial \beta} &= {\rm Diag}(f')X \cr\cr}$$An example of the equivalence of Hadamard product and diagonalization:$$\eqalign{&a = \pmatrix{a_1\\a_2},\quad&b = \pmatrix{b_1\\b_2},\quad&a&\odot&b = \pmatrix{a_1b_1\\a_2b_2} = b\odot a \cr&A = {\rm Diag}(a) = &\pmatrix{a_1&0\\0&a_2},\quad&&A&b = \pmatrix{a_1b_1\\a_2b_2} \cr&B = {\rm Diag}(b) = &\pmatrix{b_1&0\\0&b_2},\quad&&B&a = \pmatrix{a_1b_1\\a_2b_2} \cr}$$