I would suggest there are two reasons for the difference
1 Having more code to run slows down everything in exchange for doing whatever adfitional feature is provided by running the code
2 The problem is not primarily from Javascript itself other than the mistake of whoever put the library feature. I would suggest the problem is with people who are bad at computer programming writing the code in Javascript using libraries because they do not know how to program