2010-01-12

FreeBSD - How to setup ports using wget to download

The default download program for ports is using fetch. fetch is fine but my personal preference is wget. wget uses 1 connection to download, unlike my previous post that mentioned "Download ports simultaneously with multiple connections".

But wget gives extra bell and whistle like date/time of the download, log to file, continue from a partially downloaded file, rate limit of the download & others.

The below is how to setup ports to use wget to download :
Insert the following into /etc/make.conf :
FETCH_CMD=wget
FETCH_BEFORE_ARGS=-nc --progress=bar --read-timeout=60
DISABLE_SIZE=yes

-nc --> replace any files that previously downloaded, rather then clobber it
--progress=bar --> use bar to indicate download process
--read-timeout=60 --> should the internet line break in the halfway of downloading, retry for 60 seconds only. The default is 900 seconds.

Time for testing. This is an example of wget in action (along with portmanager, the ports utility) :
------------------------------------------------------------------------

fetch dri-7.4.4,2
MGPMrUpdate 0.4.1_9 command: #8 of 14  cd /usr/ports/graphics/dri && make fetch
------------------------------------------------------------------------
=> MesaLib-7.4.4.tar.bz2 doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch from http://heanet.dl.sourceforge.net/project/mesa3d/MesaLib/7.4.4/.
--2010-01-02 18:52:10--  http://heanet.dl.sourceforge.net/project/mesa3d/MesaLib/7.4.4/MesaLib-7.4.4.tar.bz2
Resolving heanet.dl.sourceforge.net... 193.1.193.66
Connecting to heanet.dl.sourceforge.net|193.1.193.66|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3375615 (3.2M) [application/x-bzip2]
Saving to: `MesaLib-7.4.4.tar.bz2'

38% [=============================>                                                 ] 1,310,592   5.23K/s   in 3m 30s

2010-01-02 18:55:40 (6.11 KB/s) - Connection closed at byte 1310592. Retrying.

--2010-01-02 18:55:40--  (try: 2)  http://heanet.dl.sourceforge.net/project/mesa3d/MesaLib/7.4.4/MesaLib-7.4.4.tar.bz2
Connecting to heanet.dl.sourceforge.net|193.1.193.66|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 3375615 (3.2M), 2065023 (2.0M) remaining [application/x-bzip2]
Saving to: `MesaLib-7.4.4.tar.bz2'

60% [++++++++++++++++++++++++++++++================>                                ] 2,030,976   4.83K/s  eta 3m 44s

Notice that the bar download process uses the mixture of symbol "+" & "=", "+" means previously downloaded sections and "=" means the current downloading process. Time for you to play with it.

Farvel !!!

2 comments:

Mahmood said...

Thanks a lot. This is what I was looking for as FETCH was completely stuck and speed was really dead slow with FETCH, using wget its awesome.

Regards,
Nasir

m@dmac said...

thanks, fetch was making my laptop way too hot to the point it would just shut down in the middle of a gnome2 install, of course there were other things involved, but fetch added to the icing, anyhow, thanks, only that you forgot/left out --follow-ftp , it would make portmaster/portmanager fail when going into distfiles folder on the servers...