How to download Ubuntu 18.04 / Bionic repository for offline

It’s time for a more technical post on Developer Daniel.

Nobody in their right mind downloads hundreds of gigabytes of data without a cause. So you’re probably reading this because you are in the following position:

1. You work in an “air-gapped” network and would like to transfer the whole Ubuntu repository into your organization’s network.
2. You’d like to have the ability to download and install packages without persistent internet connection in the future.
3. You tend to Google random queries.
4. You are a humanitarian and would like to expand the variety of Ubuntu archive mirrors in the world. Cheers!

I hope you have a fast internet connection, and at least 150GB space left on your hard disk. This may take over a day (depends on your Ubuntu repository archive).

Step 1 – Choose your favorite Ubuntu Archive Mirror Site

Go to Official Archive Mirrors for Ubuntu and choose a mirror to download from:

You can see how fast each mirror is and how stable they are.

Once you choose an official mirror, click it and copy its URL. You will need it for later on. The URL has been highlighted in red:

Step 2 – Generate or copy a sources.list file

So far, you have a mirror archive to download from. It’s time to choose which repositories you would like to download from it. That is the purpose of the sources.list file found in /etc/apt/sources.list.

For this step, either copy your local sources.list file contents, copy the following snippet or generate one on https://repogen.simplylinux.ch.

Here is an example sources.list file:

###### Ubuntu Main Repos
deb http://il.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse 
deb-src http://il.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse 

###### Ubuntu Update Repos
deb http://il.archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse 
deb http://il.archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe multiverse 
deb http://il.archive.ubuntu.com/ubuntu/ bionic-proposed main restricted universe multiverse 
deb http://il.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse 
deb-src http://il.archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse 
deb-src http://il.archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe multiverse 
deb-src http://il.archive.ubuntu.com/ubuntu/ bionic-proposed main restricted universe multiverse 
deb-src http://il.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse 

If you decided to go with a different archive mirror, you will need to substitute the URL you copied with the one in the sources.list.

Important for those who want both 32bit and 64bit support: By default, apt-mirror will only download the packages for the host’s architecture. This would mean that if you run this on a 64bit machine, only the 64bit packages will be downloaded. In order to download both, for every ‘deb …’ change it to ‘deb-amd64’ or ‘deb-i386’ (you can have both).

For example, this:

deb http://il.archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse 
deb-src http://il.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse 

Should look like this:

deb-i386 http://il.archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse 
deb-amd64 http://il.archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse 
deb-src http://il.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse 

Step 3 – Install & Configure apt-mirror

Run in the terminal to install apt-mirror:

sudo apt install apt-mirror

Configure apt-mirror with your favorite editor in /etc/apt/mirror.list. You may also copy mine:

############# config ##################
#
set base_path    /your/path/here
#
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads     20
set _tilde 0
#
############# end config ##############

# DUMP YOUR sources.list CONTENTS HERE WITHOUT THE '#'!

Make sure to update the ‘base_path’ configuration to the path where you’d like to find your downloaded repositories in. It must exist on your file system! For example, I created a ‘BionicRepo’ directory in my Downloads directory, so I configured the <base_path> to: /home/daniel/Downloads/BionicRepo.

Don’t forget to dump Step 2’s sources.list contents at the end of the mirror.list file.

* There are more configuration options which I will not walk through in this guide. For example, you can change the number of threads (‘set nthreads 20’) that download the repositories at once. In this case, there are 20 threads that perform the work. I don’t advise to change these configurations if you’re not sure with what you are doing.

Once you have that settled, it’s time for you to prove yourself how patient you can be!

Step 4 – Run the apt-mirror command

Run as the apt-mirror user:

sudo su apt-mirror
apt-mirror

Now wait until it finishes. This may take over a day (depending on the archive mirror you chose and the number of repositories you are downloading).

SANITY CHECK: If you got everything correctly up until now, you should see something like this:

And when you check the size of your <base_path>, it is increasing in size.

Step 5 – Download Complete

Congratulations! Your patience earned you a great reward: a local copy of the Ubuntu repositories!

What’s next? Before continuing onto setting up the server, check you have everything in place. Make sure the <base_path> indeed contains the amount of GB it said that will be downloaded. In my case, it was over 176GB,

Step 6 (Optional) – Set up an Apache server for easy access

If you need access from a different machine, this step will be useful.

Run the following snippet in your terminal:

sudo apt install apache2

Then configure it by running this:

sudo echo '<VirtualHost *:80>
    ServerAdmin [email protected]

    DocumentRoot /var/www
        <Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>' | sudo tee /etc/apache2/sites-enabled/000-default.conf

In order for it to work, you must link between your downloaded mirror to your web server. Moreover, since you downloaded with ‘sudo’, you need to change the owner so it can be accessed by the web server.

Simply change the </path/to/ubuntu> with the correct path. For me it was: /home/daniel/BionicRepo/mirror/il.archive.ubuntu.com/ubuntu

sudo chown -R www-data:www-data </path/to/ubuntu>
sudo ln -s </path/to/ubuntu> /var/www/ubuntu
sudo chown www-data:www-data /var/www/ubuntu

Run the following snippet (no need to change anything):

sudo rm -rf /var/www/html
sudo service apache2 restart

You should now be able to access your repo from http://localhost/ubuntu (or http://<your_ip>/ubuntu)

FAQ

Question #1: I got the following error when running apt-mirror, what do I do?:

flock() on closed filehandle LOCK_FILE at /usr/bin/apt-mirror line 214.
apt-mirror is already running, exiting at /usr/bin/apt-mirror line 217.

Answer #1: Most likely your <base_path> is NOT owned by the apt-mirror user. Try this:

sudo chown apt-mirror:apt-mirror <base_path>

Question #2: How do I check how much space I need?

Answer #2: Have at least 2GB on your machine, follow steps 1 to 4 and once you run the apt-mirror command, it will say the number of GB beyond the initial metadata that will be downloaded. The initial metadata is normally between 1 to 2 gigabytes.

Question #3: How do I use the repo locally?

Answer #3: Check this: AptGet/Offline/Repository

References

• Set Up A Local Ubuntu Mirror with Apt-Mirror
• apt-mirror is already running , exiting at /usr/bin/apt-mirror line 187.
• AptGet/Offline/Repository
  
  

Also published on Medium.