Notices by Dan Goodman (neuralreckoning@neuromatch.social)

Nice article but the social dynamics of science mean that peer review is de facto a "throughput problem". LLM usage in PR will increase because it reduces pressures on overstretched researchers. We can't stop LLM PR without addressing those pressures.

https://www.timeshighereducation.com/opinion/ai-not-peer-so-it-cant-do-peer-review

Career pressures on scientists mean we don't have time to engage with PR but feel an obligation to editors (either personally or because we're worried they'll reject our own submissions if we don't). Banning LLMs but leaving the pressures in place will just lead to concealment.

I agree with the article that the good thing about PR is not that it's an error correcting mechanism. It's actually very bad at this. Instead, it's a mechanism to get peers to engage deeply with each others' work. We can get this benefit and reduce pressure by making PR optional and post publication.

As a footnote, it's fascinating and telling how the staunchest "defenders" of peer review (journals) show the most contempt for its actual value by their willingness to flirt with the normalisation of LLM "peer" review. It's almost as if they had some other motivation. 🤷♂️

From my bsky feed - two consecutive posts. Nature Sci Rep publishes incoherent AI slop. eLife publishes a paper which the reviewers didn't agree with, making all the comments and responses public with thoughtful commentary. One of these journals got delisted by Web of Science for quality concerns from not doing peer review. Guess which one?

@elduvelle follow this thread because the commentary is almost as interesting as the paper (which also looks very cool btw):

https://bsky.app/profile/behrenstimb.bsky.social/post/3m6i6v3ydf22n

@albertcardona @elduvelle @eLife yes I'm so glad to see it! So far most eLife papers just looked like normal journal papers, but this shows what the new model can do. Brilliant stuff.

Is anarchist science possible?

As an experiment, we got together a large group of computational neuroscientists from around the world to work on a single project without top down direction. Read on to find out what happened.

The project started as a tutorial on a new technique at the @CosyneMeeting 2022. We realised that the technique was easy and cheap for anyone to use with a lot of low hanging fruit.

https://neural-reckoning.github.io/cosyne-tutorial-2022/

At the tutorial we announced a 1-2 year open research project that anyone could join, starting from the materials of the tutorial, and a few starting questions we found interesting, but with no other constraints. We were inspired by the Polymath Project in mathematics.

https://en.wikipedia.org/wiki/Polymath_Project

31 people contributed to the project, joining for monthly meetings to discuss progress. All code was publicly available throughout, and when we started writing up the work in progress was also fully public. You can see that version here: https://comob-project.github.io/snn-sound-localization/paper

Not everyone who was involved made it to the paper (didn't respond or couldn't find contact details), and not all are on Mastodon, but authors include: @marcusghosh @TomasFiers
@GabrielBena @rory

We used GitHub and Jupyter notebooks to coordinate development, with a website showing everyone's current code and results to make collaboration easier. We used @mystmarkdown and GitHub actions to automate this.

https://comob-project.github.io/snn-sound-localization/

So how did it work out? Well, some things went well and others not so well. We published a paper with our results and reflections on the process. If you're interested in spiking neural networks, sound localisation, or anarchist science, check it out:

https://www.eneuro.org/content/12/7/ENEURO.0383-24.2025

Generally, the infrastructure we built worked well, as did the monthly meetings. Starting from the tutorial was a good decision because it gave everyone a common reference and meant they could easily get started.

However, the lack of direction meant that we didn't achieve very coherent results in the end. We don't think this is a catastrophic problem, but when we try again, this is something we'd like to address. If you have thoughts or would like to be involved, get in touch!

Ultimately, we didn't achieve a scientific breakthrough in this project, but we did show that without top down direction or any specific funding, we could organise a large group of scientists to work together and publish their research in a good journal. We think that's a hopeful sign for the future!

#neuroscience #computationalneuroscience #compneuro #anarchism #science

Yeah, let's give an American company access to loads of government data and pay them a load of money to run the government for us using software that systematically fabricates false information and has nasty racial biases. That'll surely stimulate the UK economy.

https://www.bbc.co.uk/news/articles/czdv68gejm7o

Defending science in public we often talk about 'peer reviewed science'. But could this framing contribute to undermining trust in science and holding us back from improving the scientific process? How about instead we talk about the work that has received the most thorough and transparent scrutiny?

Peer review goes a step towards this in having a couple of people scrutinise the work, but there are limits on how thorough it can be and in most journals it's not transparent. Switching the framing to transparent scrutiny allows us to experiment with other models with a path to improvement.

For example, making review open to all, ongoing, and all reviews published improves this. When authors make their raw data and code open, it improves this.

It also gives us a way to criticise problematic organisations that formally do peer review but add little value (e.g. predatory journals). If their reviews are not open and observably of poor quality, then they are less 'thoroughly transparent'.

So with this framing the existence of 'peer reviewed' but clearly poor quality work doesn't undermine trust in science as a whole because we don't pin our meaning and value on an exploitable binary measure of 'peer reviewed'.

It also offers a hopeful way forward because it shows us how we can improve, and every step towards this becomes meaningful. If all we have is binary 'peer reviewed' or not, why spend more effort doing it better?

In summary, I think this new framing would be better for science, both in terms of the public perception of it, and for us as scientists.

#science #metascience #peerreview

It would be great if UK and other European universities could consider waiving requirements for a completed master's degree for students who have been de-enrolled from Harvard. It's both a good thing to do and a great opportunity to pick up some top students who would otherwise have stayed in the US

@dandean same thing but for universities and grades.

I wrote a thing for @thetransmitter. The attack on scientific infrastructure happening in the US shows that relying on any one country is not a good option for science. We need to start supporting and building international, decentralised infrastructure for science.

https://www.thetransmitter.org/policy/science-must-step-away-from-nationally-managed-infrastructure/

#science

I remember saying in 2018 that it was a shame for those real scientists elected that year (including Peter Dayan) that it happened the same year the Royal Society killed its reputation by electing Elon Musk. The Royal Society can and should undo this disastrous mistake.

Please sign @StephenCurry open letter to the president of the Royal Society below.

https://forms.gle/miDciq35oxywMEWt5

Let's remember every single company and institution that's rolling back their DEI programmes right now without even being forced to do so. The fact that they're doing it so fast means they were just waiting for the opportunity. True colours shown. Google, Meta, HHMI 😢, ...

Not a surprise for the socialists out there of course, but a stark reminder for everyone else.

Don't fall for the line that this is somehow about Russian interference. This can only distract from paying attention to the enemies of democracy who have been working quietly within the US for decades: billionaire oligarchs buying politicians and media organisations.

#SilentSunday

@elduvelle this is a great question to ask! I don't think the answer is as gloomy as you might think though.

I think you're right that there's very little point reading billionaire owned news, nor the BBC which is too scared of the government not renewing its license.

However, there's a lot of point in following and (if you're able) financially supporting independent journalism, because it can inform your actions in small ways that add up. The more we support and spread the word about independent journalism the more these small effects add up and that can have an effect.

We came so close to a left wing government with Corbyn and a few more people reading independent news could have made the difference in the first election which was only a few thousand votes from victory.

I like Novara Media for what that's worth, but choose one that you're happy with as long as it's not advertiser funded or run as a side project by famous journalists who mostly write for billionaire owned news sources (their independence is fundamentally compromised).

https://novaramedia.com/

@elduvelle it's one of the things I wanted to do with neuromatch conference. I can't stop climate change but if I can help reduce a few hundred or thousand flights by academics that's much more significant than remembering to turn off the light switch when I'm not in the room. Well, neuromatch conference doesn't run any more but SNUFA is going strong and several people have told me that it's "the" spiking neural network conference now, so maybe I'm still saving some flights.

@elduvelle this is definitely a good way to help!

The problem with the Nobel prize is not just that they've again awarded not a single woman, although they should certainly be ashamed of themselves. The problem is our deference to these fossilised patriarchal elitist institutions, reflecting broader issues in scientific culture.

It's been depressing this week too see all the excitement and respect given to this prize. It shouldn't be a surprise to us that the bequest of a wealthy weapons manufacturer, managed by a monarchic institution, makes decisions that are out of touch with our values.

Scientists should be beacons of independent and critical thought, not have their heads turned by the interests of those with wealth or status.

@tmalsburg absolutely true. There is a career path here, although it's not ideal. Write up your software as a paper, put in the time to write high quality documentation and tutorials, organise in person tutorials at conferences, etc. Make your paper highly visible on the website where you download the software and get the documentation. If your software ends up widely used it will get citations, possibly a lot, even though not everyone who uses it will cite it. All my most cited papers are software ones.

@tmalsburg some will care, and some will be swayed by enough citations, but I agree this isn't a low risk path, or one that is as valued as it should be given the impact it has on science.