Akkerman
New day, new #shell tool finding.
I was using GNU parallel before to run some simple scripts in parallel. But found out another tool from the same package: sem - semaphore for executing shell command lines in parallel.
This is a simple example:
```
for url in $urls; do
sem -j 16 curl -O "$url"
done
sem --wait
```
Run curl in 16 parallel jobs. How cool is that!
Alfred M. Szmidt
Alfred M. Szmidt
@akkerman True ture .. though #GNU make is universal. #GNU make takes the --eval / -E option so you can do the hack that way. ;-)
Akkerman
@amszmidt yes, this is an option too. But sem is a bit different, it is especially useful when you have some script already, but you need to add a pitch of parallelism into :)
івась тарасик
@amszmidt i have a problem understanding this comment, please help me?
1) i know make, i use it; i was not aware though it does paralleling out behind the scenes?!
2) shell is supposedly a more flexible tool; make can do a lot of cool things, but not everything a shell can do; for example, how would you rewrite this exact script mentioned above to do multiple curl requests with make?
3) i have a huge problem with this particular use of «just»:
https://david-alport.squarespace.com/coaching-blog/2023/2/1/just-stop-saying-just
Alfred M. Szmidt
@tivasyk make is capable of running jobs in parallell, it is also capable of having Scheme and Shell and maybe other languages embedded. Make is entirely capable of doing everything shell can do.
As for “just” .. that just means you just have a you problem.
івась тарасик
@amszmidt thank you, i'll update my definition of the «reply guy» term with this exellent example.
Alfred M. Szmidt
@tivasyk is more interested in bullying people to using his words than having a normal conversation. Here is a short example:
RSH ?= ssh
HOSTS ?= a.example b.example
hosts: $(HOSTS)
%.example:
-$(RSH) -l $(USER) $@ "$(CMD)"
make -j hosts
Runs RSH on all hosts HOSTS in parallel. This can easily be done on a single line for the curl example.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.