Using cURL to retrieve multiple URLs from a text file

curl -K params.txt

params.txt contents:

url = http://tomholland.me.uk/about
url = http://tomholland.me.uk/publications
output = file1.html
output = file2.html

Will create files named:

file1.html
file2.html

curl -K params.txt

params.txt contents:

url = http://tomholland.me.uk/{about}
url = http://tomholland.me.uk/{publications}
output = #1.html
output = #1.html

Will create files named:

about.html
publications.html

Update:

An alternative to the second method:

xargs -n1 curl -o "#1.html" < urls.txt

urls.txt contents:

http://tomholland.me.uk/{about}
http://tomholland.me.uk/{publications}

Will create files named:

about.html
publications.html

If you're running Tiger (Mac OS 10.4) or Leopard (Mac OS 10.5) then curl and xargs are there by default (I'm unsure about Jaguar or Panther).

Tagged , and .