@Munkao I know of people bringing down to KL and eastward to Kelantan. For some reason, they don't flower, or the flower falls off before their become fruit I was told by someone in PJ.
It might have evolved to only bear fruit in Perlis subtropical (instead of general Malaysian equatorial) climate, same as south Thailand. E.g. Perlis is further north along the latitude line compared to Kelantan.