Let's say I want to download something with wget but the website that has the files I need redirects to a site which automatically chooses a mirror for me (and there's no static file URL provided). Downloading from such sites works with Firefox, but how do I get real url (not only the link to the redirect page) so I could download these files with wget ?
231k 71 71 gold badges 622 622 silver badges 603 603 bronze badges asked Aug 21, 2009 at 15:03 1,451 3 3 gold badges 13 13 silver badges 18 18 bronze badgesBreakthrough's answer works, but what's the point then? Just let FF finish the download. Can you add more details to your question? I'm going to guess that you're trying to automate downloading via some sort of script, and not use wget just for gits-n-shiggles.
Commented Aug 21, 2009 at 16:40when you download with firefox right click on your download in downloads windows and select copy download link
Commented Sep 10, 2009 at 13:11@Yoopergeek: His answer helps if the file is extremely big and you want to put it into a download manager. ;)
Commented Sep 10, 2009 at 19:17@Yoopergeek: The OP also mentioned "wget" which is a linux/unix command-line tool to transfer a file without any windowsy UI at all. Tools like this come in handy when you're running a virtual machine in the cloud and need to download software. Since wget is just trying to get a file, not display advertising while it waits for a mirror to be selected, the direct url is essential.
Commented Jul 1, 2014 at 23:29Cliget addon gives you a curl command to copy and paste in the download dialog. addons.mozilla.org/en-US/firefox/addon/cliget
Commented Aug 9, 2017 at 5:24When you've started the transfer in Firefox, cancel it, and right click the download and hit "Copy download link". If you're using an older version, right click it and click on "Properties", and the link will be in the window which opens.
In Chrome - run download as normal - then go to Menu - Downloads - and you should see the direct link which was used. Or press Ctrl + J to open the window.
128 7 7 bronze badges answered Aug 21, 2009 at 15:07 Breakthrough Breakthrough 34.6k 10 10 gold badges 106 106 silver badges 153 153 bronze badges Yeah, that would be a lot simpler than what I suggested! Commented Aug 21, 2009 at 15:09my thoughts as well, arathorn. I had started typing up a response to use the 'net' tab in Firebug, but this is far simpler!
Commented Aug 21, 2009 at 15:10This does not seem to work for me on github (I tried in Firefox, Chrome, Safari). eg.: nodeload.github.com/jverzani/gWidgets2/tarball/master
Commented Apr 23, 2012 at 1:39 cmd+shift+j on Mac Commented Jun 3, 2014 at 18:53This works only if the file you are downloading is big. if the file is small, it is pain to use this method. I managed to use this method and got the link from paused download. I get exactly the same url and what I copy from a webpage. However, it is not direct link to a file. accessdata.fda.gov/scripts/sda/…
Commented Jan 30, 2023 at 16:20You can use the LiveHTTPHeaders extension to determine the actual URL of the file being downloaded. (Keep an eye on the GETs in particular.)
answered Aug 21, 2009 at 15:07 8,759 5 5 gold badges 52 52 silver badges 70 70 bronze badges that's cool idea, i'll try this. i would vote up if not my low rep ;) Commented Aug 21, 2009 at 15:08 anything for chrome? Commented Sep 16, 2015 at 10:12I know this is an old thread but wanted to give my +1 for this response. The best solution is definitely acceptable for most cases. In mine, I have an inventory web gui that I wanted to easily export all my devices from in CSV format using their export button. To make this scriptable, I took your "keep an eye on GET" advice and checked the network tab in F12 to see the file that was being downloaded by export. From there, in chrome at least, you can right click that file and do copy > copy as cURL. I substituted the device's name as a variable in the link & put it in a for loop in bash to curl
Commented Aug 7, 2019 at 15:06This is the answer you have been looking for!
There is only one fool-proof one-click solution: cliget
It gives you a curl command with headers, cookies and all, with a copy to clipboard button, right on the download dialog.
113 4 4 bronze badges answered Aug 9, 2017 at 5:29 1,010 11 11 silver badges 19 19 bronze badges Thank you! It worked like a charm Commented Sep 20, 2019 at 13:04 Great this work well. Commented Dec 16, 2019 at 16:36 Thank you! This should be the accepted answer! No other solution works for login only assets Commented Apr 28, 2022 at 10:26 I have gotten this addon for Google Chrome but I don't understand how to use it. Commented Jan 30, 2023 at 16:23@Valentyn After you download a file, you cancel the download, then you click on the cliget icon, then it gives you a choice of 3 commands to copy, with the url and request details included, which if you paste it in the terminal on the machine where you want to download the file, the file will be downloaded with all the same cookies and request and login details as in your browser.
Commented Jan 31, 2023 at 17:13While I like wget too, I use another similar tool: cURL. cURL specifically has a feature to follow redirects ( -L / --location ), and it's also free like wget. I suggest keeping both in your toolbelt; they have some complementary features.
Here's an interesting article I came across on how to use either wget or cURL to download from a site that uses cookies for authentication. There's a mention of the cURL --location feature to follow redirects. (In fact, based on no special flag mentioned for wget, it would seem to me that wget may follow redirects implicitly. )
answered Aug 21, 2009 at 16:45 Chris W. Rea Chris W. Rea 10.9k 16 16 gold badges 77 77 silver badges 95 95 bronze badgesIn those situations, First I starts the actual download and pause it and copy the URL.
answered Sep 10, 2009 at 13:17 10.7k 11 11 gold badges 44 44 silver badges 60 60 bronze badges copy it from where? Commented Sep 10, 2009 at 13:18If you pause the download in the firefox download window before it completes, you can right-click and grab the URL (have done that to fire the URL on wget sometimes).
Commented Sep 10, 2009 at 13:19You can usually right click on the file in the download dialog or page, and just choose "Copy URL". But that URL often doesn't work without the right cookie, which is where my answer comes in.
Commented Sep 26, 2022 at 4:55I usually use the firefox DownThemAll addon when there are a lot of links to download and I need to select specific URLs.
It shows the full URL and allows you to add in paused mode so you can start the download when you like.
As an additional feature, you can grab the URL and skip the download altogether.
However, if you need to download the file, DownThemAll is quite good as an accelerator.
Can't you use wget directly using the --trust-server-name flag ? It will download and save using the filename it redirects to.
answered Nov 15, 2010 at 16:55 555 1 1 gold badge 4 4 silver badges 13 13 bronze badgesYour question is a bit unclear. There are 2 urls here : the one you use, and the one that came back after redirection. The first you should know, since it originated from your machine, while the second is the one that displayed in your browser. If the question pertains to where the file is on the redirected server, then there's no way to know.
If you're trying to debug the calls issued towards the server, you can use the firefox add-on of firebug, older versions of it exist here : https://addons.mozilla.org/en-US/firefox/addons/versions/1843.
answered Sep 10, 2009 at 13:16honestly, i don't understand what's so unclear in my question, some posters below even actually understood ;)
Commented Sep 10, 2009 at 13:24If your only problem is just getting the url, then pausing the download only works for large files, unless you're very fast. You can much much simpler just find this information in the download manager's list : right-click and do Copy download link.
Commented Sep 10, 2009 at 13:55 To the downvoter: Don't waste your time here - this is a very old post and may not apply to you. Commented Aug 9, 2017 at 16:06curl --location http://domainName/fileName 2> /dev/null > fileName You can use IDM (Internet Download Manager) when you click on the not direct download link it will redirect the link and showing up the direct link.
answered Aug 21, 2009 at 16:01 virusexperts virusexpertsHave you tried to pass the redirect URL to wget? Sometimes they work for me.
Here a example from SourceForge:
$ wget http://sourceforge.net/projects/emule/files/eMule/0.49c/eMule0.49c-Installer.exe/download?use_mirror=ufpr,Centro%20de%20Computacao%20Cientifica%20e%20Software%20Livre --14:23:20-- http://sourceforge.net/projects/emule/files/eMule/0.49c/eMule0.49c-Installer.exe/download?use_mirror=ufpr,Centro%20de%20Computacao%20Cientifica%20e%20Software%20Livre Resolvendo sourceforge.net. 216.34.181.60 Connecting to sourceforge.net|216.34.181.60|:80. conectado! HTTP requisição enviada, aguardando resposta. 302 Found Localização: http://downloads.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe?use_mirror=ufpr [seguinte] --14:23:21-- http://downloads.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe?use_mirror=ufpr Resolvendo downloads.sourceforge.net. 216.34.181.59 Connecting to downloads.sourceforge.net|216.34.181.59|:80. conectado! HTTP requisição enviada, aguardando resposta. 302 Found Localização: http://ufpr.dl.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe [seguinte] --14:23:21-- http://ufpr.dl.sourceforge.net/project/emule/eMule/0.49c/eMule0.49c-Installer.exe Resolvendo ufpr.dl.sourceforge.net. 200.236.31.1, 200.17.202.1 Connecting to ufpr.dl.sourceforge.net|200.236.31.1|:80. conectado! HTTP requisição enviada, aguardando resposta. 200 OK Tamanho: 3342809 (3,2M) [application/x-msdos-program] Saving to: `eMule0.49c-Installer.exe' 100%[====================================================================================>] 3.342.809 2,35M/s in 1,4s 14:23:23 (2,35 MB/s) - `eMule0.49c-Installer.exe' saved [3342809/3342809] The version I use:
$ wget --version GNU Wget 1.10.2