Successful people do what unsuccessful people are not willing to do.

Successful people do what unsuccessful people are not willing to do. Don’t wish it were easier, wish you were better.

-Jim Rohn

Recent Posts

ML Algorithms Comparative Analysis Overview Classification Algorithms Logistic Regression Naive Bayes Classifier Regression Algorithms Linear Regression Either Classification or Regression Algorithms

Why does a transformer not use an activation function following the multi-head attention layer?

why in the transformer model from the "Attention is all you need" paper there is no activation applied after both the multihead attention layer and to the residual connections. It seems to me that the