Conversation

Notices

1. Embed this notice
   Thorsten Leemhuis (acct. 1/4) (kernellogger@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:47 JST Thorsten Leemhuis (acct. 1/4)

   linus-next: improving functional testing for to-be-merged [#linux #kernel] pull requests

   https://lore.kernel.org/all/ZxZ8MStt4e8JXeJb@sashalap/

   "'[…] Linus Torvalds expressed concerns about the quality of testing that code receives before he pulls it. The subsequent discussion side-tracked to the testability of linux-next, but we didn't directly address Linus's original concern about pre-pull testing quality.

   In an attempt to address the concerns, we're trying out a new "linus-next"
   tree […]"

   #LinuxKernel

   • Embed this notice
     Kees Cook :tux: (kees@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:40 JST Kees Cook :tux:

     in reply to

     • Lorenzo Stoakes
     • Vlastimil Babka

     @ljs @kernellogger @vbabka Why? Just use linux-next. Nothing should go in it that's not ready to go to Linus. Who is it that isn't testing -next? I'm always testing there. All the CIs I see reporting to lkml are testing -next. I genuinely didn't understand what's missing. -next is for testing. Do people need to be reminded to test on _copies_ of their data/images/workloads?

     James Morris likes this.
   • Embed this notice
     Lorenzo Stoakes (ljs@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:41 JST Lorenzo Stoakes

     in reply to

     • Kees Cook :tux:
     • Vlastimil Babka
     @kernellogger @vbabka @kees hey we could even have a
     
     next-for-humans
     next-for-bots
     
     ;)
   • Embed this notice
     Vlastimil Babka (vbabka@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:42 JST Vlastimil Babka

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     @ljs @kees @kernellogger I think the "sometimes merges new series too quickly" is perceived as the issue especially if it breaks -next and prevents testing there. Last LSF Andrew said he might create mm-experimental as another branch that would not be in -next so maybe that would move in the direction Thorsten would like too?
   • Embed this notice
     Thorsten Leemhuis (acct. 1/4) (kernellogger@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:42 JST Thorsten Leemhuis (acct. 1/4)

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     • Vlastimil Babka

     @vbabka @kees @ljs

     yeah, sounds like it; and bots could even process that mm-experimental branch, like they do for quite a few other subsystem trees not included in -next.

   • Embed this notice
     Lorenzo Stoakes (ljs@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:43 JST Lorenzo Stoakes

     in reply to

     • Kees Cook :tux:
     • Vlastimil Babka
     @kernellogger @kees @vbabka I mean some series take weeks and weeks and weeks to get through review.
     
     So what you're saying is - during all of those weeks - we must not have build bots on the series, we must not have anything doing even ostensibly small levels of testing, we just wait until rc and ship it as is and that's 'safer' and will make people 'less scared'.
     
     OK.
   • Embed this notice
     Lorenzo Stoakes (ljs@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:43 JST Lorenzo Stoakes

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     • Vlastimil Babka
     @kernellogger @kees @vbabka to be clear I am not a fan of the 'automerge' stuff, but I do think once something has at least sufficient review it should go in to -next, even if it might have unexpected later rounds of review.
     
     Andrew won't take anything that has outstanding review on it btw. It is literally only the stuff that truly looks like it's going in.
     
     But I concede I'd like there to be at least 1 tag and a day or 2 to make sure no obvious other objection before moving to -next.
     
     The basically 'wait until we are really really sure there's nothing more before subjecting to testing' take though, yeah profoundly disagree.
     
     What I've been finding with my series is that it goes to Linus and _suddenly_ you get a bunch of reports and are doing 12 hour days to fix things.
     
     If you want less 'you ate my data!' I'd say test sooner.
     
     Anybody testing -next and expecting serious stability is being silly even if all of your conditions were met, it's very fresh unstabilised code you can't rely on it.
     
     And I literally run an rc as my main system atm btw...
   • Embed this notice
     Thorsten Leemhuis (acct. 1/4) (kernellogger@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:44 JST Thorsten Leemhuis (acct. 1/4)

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     • Vlastimil Babka

     @ljs @kees @vbabka

     > I think the issue is that more people need to test against -next.

     Avoiding "this might eat my data" fears for the testers is very important detail here to realize that – having patches in there that come from a "unstable" branch is enough to scare people away when it comes to mm or filesystems.

     So chancing that name might already help.

   • Embed this notice
     Lorenzo Stoakes (ljs@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:44 JST Lorenzo Stoakes

     in reply to

     • Kees Cook :tux:
     • Vlastimil Babka
     @kernellogger @kees @vbabka who is testing unstable unreleased kernel code while also being scared of 'having their data eaten'??
     
     The whole point of -next is 'here is a snapshot of what is probably coming next, please test it'.
     
     rc's might eat your data too, the whole point is for testing to catch this stuff
   • Embed this notice
     Thorsten Leemhuis (acct. 1/4) (kernellogger@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:44 JST Thorsten Leemhuis (acct. 1/4)

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     • Vlastimil Babka

     @ljs @kees @vbabka

     eating data is always a risk spectrum (even in stable release there is a risk that it happens) -- and depending on how risky something looks people then decide what they use/test: stable, stable-rc, mainline outside the merge window (that's me currently), mainline all the time, -next.

     The less likely the "might eat data" risk is perceived in -next, the more likely people will be willing to help with testing it.

   • Embed this notice
     Lorenzo Stoakes (ljs@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:45 JST Lorenzo Stoakes

     in reply to

     • Kees Cook :tux:
     • Vlastimil Babka
     @kernellogger @kees @vbabka they are ostensibly reviewed by Andrew.
     
     I mean thanks for trying to make stuff get _less_ tested before rc though...
   • Embed this notice
     Lorenzo Stoakes (ljs@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:45 JST Lorenzo Stoakes

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     • Vlastimil Babka
     @kernellogger @kees @vbabka I don't necessarily agree with the process in mm as it stands but this is how it works right now, that is a lack of objection and Andrew not finding major flaws = Andrew in effect approves, and unless it's a relative newcomer or otherwise he has reason not to want it, the series will get merged.
     
     So it's representative of what will be in the next merge window (+ hotfixes not yet upstreamed for rc).
     
     I'd like there to be more of a 'needs a tag from at least someone' rule in mm, but can't control that. It adds workload to people who have to act fast to stop stuff getting pulled in.
     
     If we limited it to mm-stable or something then -next would be _miles_ behind what is going into the next merge window, and you'd also _not_ be stabilising as while not enough people test -next NOBODY tests mm-unstable so it'd become a pretty useless tree.
     
     Again, I think the issue is that more people need to test against -next.
   • Embed this notice
     Thorsten Leemhuis (acct. 1/4) (kernellogger@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:46 JST Thorsten Leemhuis (acct. 1/4)

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     • Vlastimil Babka

     @vbabka and @ljs, that "Are people putting things in linux-next that they don't expect to send to Linus? That seems like the greater problem." from @kees reminded me of a question you might be able to help out with:

     From a quick look it seems to me that the "mm-unstable" branch is in -next (via "mm-everything"). Does that contain stuff for the next merge window only, or more experimental stuff as well? It looks like the latter to me.

   • Embed this notice
     Lorenzo Stoakes (ljs@social.kernel.org)'s status on Wednesday, 23-Oct-2024 01:24:46 JST Lorenzo Stoakes

     in reply to

     • Kees Cook :tux:
     • Vlastimil Babka
     @kernellogger @vbabka @kees what does 'experimental' mean?
     
     mm-unstable is everything that _appears_ to be going to Linus because nobody objected to it yet but some stuff might not end up going because it's got issues.
     
     I'm not sure how you're supposed to differentiate between stuff that's eventually going to get reviewed to the point of not being submitted vs. stuff that'll go?
     
     I think this is a bad take to be honest.
     
     Next _should_ contain stuff we _expect_ will go to Linus, which _is_ all of that.
     
     The issue is that people aren't bloody testing next!
   • Embed this notice
     Thorsten Leemhuis (acct. 1/4) (kernellogger@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:46 JST Thorsten Leemhuis (acct. 1/4)

     in reply to

     • Kees Cook :tux:
     • Lorenzo Stoakes
     • Vlastimil Babka

     @ljs @kees @vbabka

     Well, good points, but fwiw, afaiui only patches that *were* reviewed by one of the official maintainers are supposed to be included in -next. To quote Stephen from https://lore.kernel.org/linux-next/20240716083116.27f179bd@canb.auug.org.au/

     "'You will need to ensure that the patches/commits in your tree/series have
     been:
     […]
     * reviewed by you (or another maintainer of your subsystem tree),
     * successfully unit tested, and
     * destined for the current or next Linux merge window."'

   • Embed this notice
     Thorsten Leemhuis (acct. 1/4) (kernellogger@fosstodon.org)'s status on Wednesday, 23-Oct-2024 01:24:47 JST Thorsten Leemhuis (acct. 1/4)

     in reply to

     • Kees Cook :tux:

     2/ @kees replied to the "linus-next" proposal from Sasha and raised a few points I fully agree with, as that proposal felt a bit off for me.

     https://lore.kernel.org/all/792F4759-EA33-48B8-9AD0-FA14FA69E86E@kernel.org/

     "'Are people putting things in linux-next that they don't expect to send to Linus? That seems like the greater problem.

     […]

     Why not just use linux-next? […]

     […] have a bot that replies to all PRs with a health check, and Linus can pull it if he thinks it looks good. […]"

     #Linux #kernel #LinuxKernel