lazyvim, nvchad, etc are basically just configs that other people did.
So, you can just pick the one you like the most, and tweak it to your liking.
LazyVim happened to be the one that worked the best for me out of the box (and I liked how the files were organized). So I used it as a starting point for my config.
If you want to start with something more minimalist to build your own config, have a look at kickstart.nvim