Batch file question

[Phoenix64d]

Howdy,

Trying to design a batch file that will go to an internet site, download a file to a specific folder and replace all files with the same name within that folder. I've only made batches to do very simple things, and the more complicated commands are lost on me.

Is there anyway to do this?

[Ace]

Is there anyway to do this?

Sounds easy; would be far easier on Linux (as everything is...)

I'm not aware of a wget equivalent for windows, so I'd probably go and download wget sources from one of the wget ports for windows projects (google it). Of course, you'll have to piss around compiling them in standard ****e windows' fashion (probably Visual Studio etc!!). But after you've got the wget command working, the rest is easy.

wget http://www.mysite.com/my/file.whatever store/here/file.whatever

replace all files with the same name within that folder
If you mean you want to overwrite any existing file with the same name as that you're downloading, then include the -r option:

wget -r http://www.mysite.com/my/file.whatever store/here/file.whatever

Without that you'd get file.whatever and file.whatever.1 etc

Maybe you'd want to turn off command echoing too, and have wget spit out a bit less info (to make it silent use the -q flag or stick " > NULL" on the end):

@echo off
wget --no-verbose -r http://www.mysite.com/my/file.whatever store/here/file.whatever

[jdrom17]

Don't know if you have to use wget, but I don't believe there's any built-in command to do it. So you're going to have to use some sort of 3rd party app instead.

[Ace]

Don't know if you have to use wget
True, I should have mentioned that there will be many other solutions. I'm just running through wget because I use Linux, and on Linux such a task with wget is a daily event that barely takes seconds to perform ;) Indeed, it may be preferable to use an app made for Windows rather than a port.

Indeed, were I in Winblows, I would just boot to Linux or use Cygwin, rather than compiling the port (because compiling things on Winblows is a PitA in itself!! - VS etc, compared to a quick g++ command or a makefile).

Having said all that, a quick google returned a version of
wget fully compiled (http://users.ugent.be/~bpuype/wget/) for Windows as an exe, so you can simply stick it somewhere in your PATH and use the wget command as above :)

[Phoenix64d]

Lol, Ace just has the answers to all of my questions lately. It's funny, b/c a friend of mine sent me wget the same day I read this post. Thanks a lot. Maybe I can ask another question about Wget then

I'm trying to use Wget to automatically download updated drivers, programs, virus definition files, etc. The problem is that these updates change name (say cpuz150.zip vs cpuz149.zip), and they are not always hosted on the same page.

What I had in mind was telling Wget to download all links from a website with a certain name in the file. For example, if we take the CPUZ example above, I could tell wget to download cpuz150 as opposed to 149 by telling it to look for file cpuz1?? with a .zip extension.

I know Im probably pushing it. Think Wget can do this?

[Ace]

Hehe, no problem.

That may be rather more difficult, depending. wget itself supports wildcards, but only when possible. In other words, for HTTP sites that support directory listing, and for FTP too. If one of those conditions are met, you should be able to use something like:

wget http://www.site.com/files/cpu1*

Plus, you can use the -A flag to specify an extension. Haven't tried myself, but something like this should probably work:

wget -r -A.zip http://www.site.com/files/cpu1*

Or if it doesn't like that, try something like this (reckon this should work!):

wget -r -Acpu1*.zip http://www.site.com/files/

Depending on the structure of the site, you may want to revise the use of the -r recursive flag (I've included it so that any files in the dir you're downloading to with the same name as files you're grabbing are overwritten, but you could just as easily add a little command to remove such files beforehand, or move them somewhere else).

If the site isn't FTP and doesn't support HTTP dir listing, then you've got a slight problem :) Unless you can essentially predict what the next version number may be and hence what the filename may be, I'm not sure.

e.g. in the case of cpuz149.zip, you could assume that the next version will likely retain the same filename format, and try incrementing the number (150) and trying a wget, try incrementing again and retry wget, try incrementing the second digit (minor version number), the first digit and reset the other two (primary version number)... have a set number of tests before the script gives up and determines there's probably no new version out. But that's rather messy :)

The only other way would be if the site in question has a permanent link to the latest version that you can take advantage of. Don't forget: wget can download http files, and you could set the script to search through the file for a specific string, extract it, and use it in the wget command.

[Phoenix64d]

I am *this* close to getting the hang of this. Thanks for the help so far.

I'm getting an error about wget not supporting wildcards when hitting HTTP. In most cases, Ive just worked around it by finding permanent links like you said; but I don't know why I am getting this error, when the manual specifically says you can use wildcards in these circumstances.

Also, I am having a little issue with how wget is saving files. Although I can get it to not recreate the site directory structure, Ill be damed if can set it to save as filenames and directories of my choice.

------------------
ANSWERED
------------------
Figured out how to save as specific files. I need this command "-O./ 'file' " to save to a document named 'file.'

I think I'm also resolving the wildcard issue. I can't use a wildcard in an address, any more than I could in a browser. Wildcards only come into play when wget has identified files in a website that shows directory structure. So, Im trying to download Piriform defraggler and all future versions, but cannot b/c the download address changes depending on the current version and strangely Wget won't download the .exe (only the .zip) from the root directory download page (http://www.defraggler.com/download/builds)

Do you know of any clever way I might pull this off? I've run out of ideas for now.

[Ace]

Glad to see you're getting along ok :) Yes, wget only supports wildcards when the site supports HTTP directory listing; i.e., you get a a directory structure of files when you visit in a browser, as opposed to standard web pages.

I seemed to be able to download the .exes fine:

ace@ace5:~$ wget -r http://download.piriform.com/dfsetup109.exe
--2009-04-18 04:07:56-- http://download.piriform.com/dfsetup109.exe
Resolving download.piriform.com... 72.21.211.148
Connecting to download.piriform.com|72.21.211.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 805120 (786K) [application/x-msdos-program]
Saving to: `download.piriform.com/dfsetup109.exe'

100%[======================================>] 805,120 228K/s in 4.4s

2009-04-18 04:08:02 (179 KB/s) - `download.piriform.com/dfsetup109.exe' saved [805120/805120]

One way round the non-directory listing for the piriform site would be to use wget to download the page http://download.piriform.com (XML document) and parse it. It would likely be easiest to use some sort of XML parser for the task rather than coding yourself.

<Contents>
<Key>dfsetup109.exe</Key>
<LastModified>2009-04-14T08:11:30.000Z</LastModified>
<ETag>&quot;4642cf0ab33cbb0e2109a4dc4bb6200b&quot;</ETag>
<Size>805120</Size>
<StorageClass>STANDARD</StorageClass>
</Contents>

It's a case of parsing the Key fields, looking for your prefix dfsetup*, & extracting the highest following 3 digits, or perhaps parsing the LastModified fields, sorting, and then using the Key field of the latest one to download from.

This would probably entail using, for example, Java/C++/Python/Perl/whatever language you're comfortable XML parsing libraries, and building a little app. Mind you, you could also code in the downloading too ;) but this is a bit of effort.

edit: I guess another way would be to "guess" some login details for ftp://piriform.com, which would give you full wildcard support, but this I would definitely not advise :p