@Rasp That's cool.
It seems there's these things called imaginal disks that develop throughout their life, coming into play during metamorphosis, which make up the framework of their new body.
Might be possible that some memories are stored there, or else... the soup remembers. 👁️
As for why they do it, I think about it like this. Why do we get horny? We're just vibing growing up, then our body changes and says "heyyyy so uhhh..." and bam, new thoughts unlocked.
It's just an urge. The caterpillar grows up and a hormone tells them to fucking liquefy themselves, I suspect.