@TrevorGoodchild @sun Extremely unlikely without using accounting tricks. There is no publicly known scheme for training models that size that doesn't require a ton of compute.

Either they have a new scheme, which has taken a lot of research and time to find out, or they are using some local chip which took a ton of time and money to make.