I can make time to look into it some time between now and the end of this weekend unless mattly beats me to it. Unsetting pandoc variables should do it (the man page has a nice Variables for HTML section, btw).
Failing that, we'd just need to add some specificity to our own color/background-color rules, but better to not have to do that if we can avoid it, since it's easier to tell when pandoc options change between pandoc versions than tracking whether changes to its CSS output break our overrides.