Downloading multiple files from username/password protected website

gerryc89 · Aug 29, 2012

Hi all,

Im hoping someone can help me here. I have just left university but want to download all the files from our resource folder. Basically its 4 years of notes! They are very valuable and I could manually save them, but it would take ages.

Its obviously password and username protected. Is there a program/script/app I can use to download these?

They are in multiple formats - pdf, jpeg, word, excel etc

Is there a way of doing this?

I would be most grateful for any guidance/help

Gerry

Raz0rEdge · Aug 29, 2012

How do you access these files? Is it over the network? Network share? or something else? Based on method of access, we can suggest the fastest way of getting the files..

gerryc89 · Aug 29, 2012

Hi Raz0rEdge

Thanks for your quick reply!

I access it online on a webpage, so its a website that we log into. The site ends in .aspx if that sheds further light on it?

Many thanks for your help, much appreciated!

Gerry

Raz0rEdge · Aug 29, 2012

Well first and foremost if this information is available to you to keep, you might want to contact the person who put it there and see if they can transfer it to you in a ZIP file or a USB flash drive or something.

if the website is the only way, scripting it gets a little annoying unless you can provide the username/password directly in the URL of a file and have it be downloaded.

vansmith · Aug 29, 2012

Raz0rEdge said:
if the website is the only way, scripting it gets a little annoying unless you can provide the username/password directly in the URL of a file and have it be downloaded.

Which is probably unlikely if the webmasters have any concern about the security of their system. I can't imagine that may web applications allow someone to pass a password as plain text anymore (did they ever?).

The problem with scripting this, as Raz0rEdge alluded to, is authentication. Getting the actual files themselves is comparatively easy but is impossible if you can't get past the authentication process. I'm going to guess that the university doesn't make an API available for authenticating to these services since there's no point so you my be out of luck here. Instead, you might want to look into browser extensions that can download all the files on a webpage. For example, something like DownloadThemAll for Firefox might do the trick.

gerryc89 · Aug 29, 2012

Yes, however the problem is that it is from multiple sources. Probably around 20 lecturers! I may ask IT at my university and see if they can help me.

I see what you mean, the username/password wont be in the URL so it will probably end up being as much work as just doing it manually!

Thanks for your help

gerryc89 · Aug 29, 2012

Hi Vansmith,

I tried that first, it was continuously asking for username/password. My keychain held the info but hitting ok a dozen times and it still didn't work.

I think you're right, that is the prob! There seem to many programs for doing this, its the issue with authentication here that is the killer!

Thanks for your help

vansmith said:
Which is probably unlikely if the webmasters have any concern about the security of their system. I can't imagine that may web applications allow someone to pass a password as plain text anymore (did they ever?).

The problem with scripting this, as Raz0rEdge alluded to, is authentication. Getting the actual files themselves is comparatively easy but is impossible if you can't get past the authentication process. I'm going to guess that the university doesn't make an API available for authenticating to these services since there's no point so you my be out of luck here. Instead, you might want to look into browser extensions that can download all the files on a webpage. For example, something like DownloadThemAll for Firefox might do the trick.

IvanLasston · Aug 29, 2012

Kinda sounds like a sharepoint site. Does it have a ribbon like interface?

Anyway - if it is sharepoint there is no easy way as far as I can tell to do a batch copy. Seems if you have outlook connected or if you access to windows explorer there might be a way to do a batch copy.
how to download folder (with all documents) from Document library?

Anything you can tell us about what the actual web page / server is - the more we can help.

Edit: Man I'm slow at posting

gerryc89 · Aug 29, 2012

HI Ivan,

Yes! It does. I took a screenshot of an example page. Does this help?

Gerry

IvanLasston said:
Kinda sounds like a sharepoint site. Does it have a ribbon like interface?

Anyway - if it is sharepoint there is no easy way as far as I can tell to do a batch copy. Seems if you have outlook connected or if you access to windows explorer there might be a way to do a batch copy.
how to download folder (with all documents) from Document library?

Anything you can tell us about what the actual web page / server is - the more we can help.

Edit: Man I'm slow at posting

IvanLasston · Aug 29, 2012

That does look like sharepoint. The suggestion of Open in Windows Explorer does work - and then you can grab everything and copy it - but that would mean you need to find a Windows machine to do that.

If you are using Internet Explorer and Windows -> Select the Library tab -> you'll see Open with Explorer as one of the choices in that ribbon.

Either use a Virtual Machine, Bootcamp, or bum somebody's computer - because those options are not available on OSX as far as I can tell.

Downloading multiple files from username/password protected website

gerryc89

Raz0rEdge

Well-known member

gerryc89

Raz0rEdge

Well-known member

vansmith

Senior Member

gerryc89

gerryc89

IvanLasston

gerryc89

IvanLasston

Shop Amazon