apt-offline 1.8.2
I am pleased to announce the release of apt-offline version 1.8.2
This release has many bug fixes along with a long standing issue of signature validation
2017 - The year of realization and change
Back in 2017, the bug was reported that apt-offline did not validate apt meta Packages files. apt-offline was only doing a signature validation for the Release files but did no validation of the apt meta Packages files, which had their checksums listed in the Release files. This validation was completely missing in apt-offline and gave the user the wrong impression that validation was in place.
I had hoped to fix this issue soon when it was reported, to have it part of the next Debian Stable release. But that never happened. On the contrary, I think 2 stable releases happened in between. And now it is 2020.
2017 was a year to spend a large chunk of my time on real life issues, for good. I realized that it is important to always give precedence to personal life, fix issues, set realistic priorities, spend time on realizing the happenings around, get life rolling smooth and then come back to work. This helps sustain in the longer run. Otherwise, with no self, everything can fall apart catastrophically.
From that phase, I learned many things. I now have much more respect for people who really have been successful at committing a large amount of their time on a volunteer project like Debian. Having myself gone through the time crunch phase, I can only imagine how many of the fellow DDs manage their time, sustainably, over the years. There are many folks I have seen active for more than a decade and they still rock.
1.8.2 release
Because the apt meta validation was a major issue, I have decided to run through the workflow and explain how apt-offline reacts to invalid tampered data. Below are konsole captures, with snipped output, where not very relevant.
rrs@priyasi:/var/tmp/Debian-Build/Result$ sudo apt-offline set /tmp/set.uris
Gathering details needed for 'update' operation
Gathering details needed for 'upgrade' operation
16:33 β ΰ₯ βΊ π
One item to pay attention to, in this step, is some of the errors that are reported. Not all repository admins enable all the apt meta data available on their mirrors. This is commonly seen for localization related files. Similarly, not all compression types are available on all the repository servers. Some may only have .xz based meta files hosted while others may have .gz ones. So, for apt-offline, which has to bridge the gap of the offline <=> online setup, there is more work.
For compression types, apt-offline cycles through the known list of types. Only if, after cycling through all the known compression types, if the return is still a 404, then we error out.
Similarly, for localization related meta, we do the same cycling. But in addition to that, there is the possibility that the repository admin may not have enabled the localization data to be served at all. In that case, apt-offline ultimately will report and error.
And that is what is shown below. Because I see them not breaking the functionality, I treat them as non-fatal errors.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
|
rrs@priyasi:/var/tmp/Debian-Build/Result$ apt-offline get /tmp/set-trimmed.uris --bundle /tmp/set.zip --threads 5
Fetching APT Data
WARNING: If you are on a slow connection, it is good to
WARNING: limit the number of threads to a low number like 2.
WARNING: Else higher number of threads executed could cause
WARNING: network congestion and timeouts.
Downloading http://deb.debian.org/debian/dists/testing/Release.gpg
Downloading http://deb.debian.org/debian/dists/testing/Release
Downloading http://deb.debian.org/debian/dists/testing/InRelease
Downloading http://deb.debian.org/debian/dists/unstable/Release.gpg
Downloading http://deb.debian.org/debian/dists/unstable/Release
http://deb.debian.org/debian/dists/unstable/Release.gpg done
Downloading http://deb.debian.org/debian/dists/unstable/InRelease
http://deb.debian.org/debian/dists/testing/Release.gpg done
Downloading http://deb.debian.org/debian/dists/experimental/Release.gpg
http://deb.debian.org/debian/dists/unstable/Release done
Downloading http://deb.debian.org/debian/dists/experimental/Release
http://deb.debian.org/debian/dists/testing/InRelease done
Downloading http://deb.debian.org/debian/dists/experimental/InRelease
http://deb.debian.org/debian/dists/testing/Release done
Downloading http://deb.debian.org/debian/dists/testing/main/source/Sources.xz
http://deb.debian.org/debian/dists/unstable/InRelease done
Downloading http://deb.debian.org/debian/dists/testing/non-free/source/Sources.xz
http://deb.debian.org/debian/dists/experimental/Release.gpg done
Downloading http://deb.debian.org/debian/dists/testing/contrib/source/Sources.xz
http://deb.debian.org/debian/dists/experimental/InRelease done
Downloading http://deb.debian.org/debian/dists/testing/main/binary-amd64/Packages.xz
http://deb.debian.org/debian/dists/experimental/Release done
Downloading http://deb.debian.org/debian/dists/testing/main/binary-i386/Packages.xz
http://deb.debian.org/debian/dists/testing/contrib/source/Sources.xz done
Downloading http://deb.debian.org/debian/dists/testing/main/binary-all/Packages.xz
http://deb.debian.org/debian/dists/testing/non-free/source/Sources.xz done
Downloading http://deb.debian.org/debian/dists/testing/main/i18n/Translation-en_IN.xz
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/main/i18n/Translation-en_IN.lzma
Downloading http://deb.debian.org/debian/dists/testing/main/i18n/Translation-en.xz
http://deb.debian.org/debian/dists/testing/main/binary-all/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/main/i18n/Translation-en_US.xz
http://deb.debian.org/debian/dists/testing/main/source/Sources.xz done
Downloading http://deb.debian.org/debian/dists/testing/main/Contents-amd64.xz
http://deb.debian.org/debian/dists/testing/main/i18n/Translation-en.bz2 done
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/main/i18n/Translation-en_US.lzma
Downloading http://deb.debian.org/debian/dists/testing/main/Contents-i386.xz
Downloading http://deb.debian.org/debian/dists/testing/main/Contents-all.xz
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/main/Contents-all.lzma
Downloading http://deb.debian.org/debian/dists/testing/non-free/binary-amd64/Packages.xz
http://deb.debian.org/debian/dists/testing/non-free/binary-amd64/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/non-free/binary-i386/Packages.xz
http://deb.debian.org/debian/dists/testing/non-free/binary-i386/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/non-free/binary-all/Packages.xz
http://deb.debian.org/debian/dists/testing/non-free/binary-all/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/non-free/i18n/Translation-en_IN.xz
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/non-free/i18n/Translation-en_IN.lzma
Downloading http://deb.debian.org/debian/dists/testing/non-free/i18n/Translation-en.xz
http://deb.debian.org/debian/dists/testing/non-free/i18n/Translation-en.bz2 done
Downloading http://deb.debian.org/debian/dists/testing/non-free/i18n/Translation-en_US.xz
http://deb.debian.org/debian/dists/testing/main/binary-i386/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/non-free/Contents-amd64.xz
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/non-free/i18n/Translation-en_US.lzma
Downloading http://deb.debian.org/debian/dists/testing/non-free/Contents-i386.xz
http://deb.debian.org/debian/dists/testing/non-free/Contents-i386.gz done
Downloading http://deb.debian.org/debian/dists/testing/non-free/Contents-all.xz
http://deb.debian.org/debian/dists/testing/non-free/Contents-amd64.gz done
http://deb.debian.org/debian/dists/testing/main/binary-amd64/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/contrib/binary-amd64/Packages.xz
Downloading http://deb.debian.org/debian/dists/testing/contrib/binary-i386/Packages.xz
http://deb.debian.org/debian/dists/testing/contrib/binary-amd64/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/contrib/binary-all/Packages.xz
http://deb.debian.org/debian/dists/testing/contrib/binary-i386/Packages.xz done
http://deb.debian.org/debian/dists/testing/contrib/binary-all/Packages.xz done
Downloading http://deb.debian.org/debian/dists/testing/contrib/i18n/Translation-en_IN.xz
Downloading http://deb.debian.org/debian/dists/testing/contrib/i18n/Translation-en.xz
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/non-free/Contents-all.lzma
Downloading http://deb.debian.org/debian/dists/testing/contrib/i18n/Translation-en_US.xz
http://deb.debian.org/debian/dists/testing/contrib/i18n/Translation-en.bz2 done
Downloading http://deb.debian.org/debian/dists/testing/contrib/Contents-amd64.xz
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/contrib/i18n/Translation-en_IN.lzma
Downloading http://deb.debian.org/debian/dists/testing/contrib/Contents-i386.xz
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/contrib/i18n/Translation-en_US.lzma
Downloading http://deb.debian.org/debian/dists/testing/contrib/Contents-all.xz
http://deb.debian.org/debian/dists/testing/contrib/Contents-amd64.gz done
http://deb.debian.org/debian/dists/testing/contrib/Contents-i386.gz done
ERROR: Giving up on URL http://deb.debian.org/debian/dists/testing/contrib/Contents-all.lzma
http://deb.debian.org/debian/dists/testing/main/Contents-i386.gz done
http://deb.debian.org/debian/dists/testing/main/Contents-amd64.gz done
81 / 81 items: [##############################] 100.0% of 101 MiB
Downloaded data to /tmp/set.zip
ERROR: Some items failed to download. Downloaded data may be incomplete
ERROR: Please run in verbose mode to see details about failed items
16:38 β ΰ₯ βΉ π=> 100
|
First, lets unpack the archive file.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
rrs@priyasi:/var/tmp/Debian-Build/Result$ cd /tmp/
16:39 β ΰ₯ βΊ π
rrs@priyasi:/tmp$ mkdir set-folder
16:39 β ΰ₯ βΊ π
rrs@priyasi:/tmp$ cd set-folder/
16:39 β ΰ₯ βΊ π
rrs@priyasi:/tmp/set-folder$ unzip ../set.zip
Archive: ../set.zip
inflating: deb.debian.org_debian_dists_unstable_Release.gpg
inflating: deb.debian.org_debian_dists_testing_Release.gpg
inflating: deb.debian.org_debian_dists_unstable_Release
inflating: deb.debian.org_debian_dists_testing_InRelease
inflating: deb.debian.org_debian_dists_testing_Release
inflating: deb.debian.org_debian_dists_unstable_InRelease
inflating: deb.debian.org_debian_dists_experimental_Release.gpg
inflating: deb.debian.org_debian_dists_experimental_InRelease
inflating: deb.debian.org_debian_dists_experimental_Release
inflating: deb.debian.org_debian_dists_testing_contrib_source_Sources.xz
inflating: deb.debian.org_debian_dists_testing_non-free_source_Sources.xz
inflating: deb.debian.org_debian_dists_testing_main_binary-all_Packages.xz
inflating: deb.debian.org_debian_dists_testing_main_source_Sources.xz
inflating: deb.debian.org_debian_dists_testing_main_i18n_Translation-en.bz2
inflating: deb.debian.org_debian_dists_testing_non-free_binary-amd64_Packages.xz
inflating: deb.debian.org_debian_dists_testing_non-free_binary-i386_Packages.xz
inflating: deb.debian.org_debian_dists_testing_non-free_binary-all_Packages.xz
inflating: deb.debian.org_debian_dists_testing_non-free_i18n_Translation-en.bz2
inflating: deb.debian.org_debian_dists_testing_main_binary-i386_Packages.xz
inflating: deb.debian.org_debian_dists_testing_non-free_Contents-i386.gz
inflating: deb.debian.org_debian_dists_testing_non-free_Contents-amd64.gz
inflating: deb.debian.org_debian_dists_testing_main_binary-amd64_Packages.xz
inflating: deb.debian.org_debian_dists_testing_contrib_binary-amd64_Packages.xz
inflating: deb.debian.org_debian_dists_testing_contrib_binary-i386_Packages.xz
inflating: deb.debian.org_debian_dists_testing_contrib_binary-all_Packages.xz
inflating: deb.debian.org_debian_dists_testing_contrib_i18n_Translation-en.bz2
inflating: deb.debian.org_debian_dists_testing_contrib_Contents-amd64.gz
inflating: deb.debian.org_debian_dists_testing_contrib_Contents-i386.gz
inflating: deb.debian.org_debian_dists_testing_main_Contents-i386.gz
inflating: deb.debian.org_debian_dists_testing_main_Contents-amd64.gz
16:39 β ΰ₯ βΊ π
rrs@priyasi:/tmp/set-folder$ ls
deb.debian.org_debian_dists_experimental_InRelease deb.debian.org_debian_dists_testing_main_Contents-i386.gz
deb.debian.org_debian_dists_experimental_Release deb.debian.org_debian_dists_testing_main_i18n_Translation-en.bz2
deb.debian.org_debian_dists_experimental_Release.gpg deb.debian.org_debian_dists_testing_main_source_Sources.xz
deb.debian.org_debian_dists_testing_contrib_binary-all_Packages.xz deb.debian.org_debian_dists_testing_non-free_binary-all_Packages.xz
deb.debian.org_debian_dists_testing_contrib_binary-amd64_Packages.xz deb.debian.org_debian_dists_testing_non-free_binary-amd64_Packages.xz
deb.debian.org_debian_dists_testing_contrib_binary-i386_Packages.xz deb.debian.org_debian_dists_testing_non-free_binary-i386_Packages.xz
deb.debian.org_debian_dists_testing_contrib_Contents-amd64.gz deb.debian.org_debian_dists_testing_non-free_Contents-amd64.gz
deb.debian.org_debian_dists_testing_contrib_Contents-i386.gz deb.debian.org_debian_dists_testing_non-free_Contents-i386.gz
deb.debian.org_debian_dists_testing_contrib_i18n_Translation-en.bz2 deb.debian.org_debian_dists_testing_non-free_i18n_Translation-en.bz2
deb.debian.org_debian_dists_testing_contrib_source_Sources.xz deb.debian.org_debian_dists_testing_non-free_source_Sources.xz
deb.debian.org_debian_dists_testing_InRelease deb.debian.org_debian_dists_testing_Release
deb.debian.org_debian_dists_testing_main_binary-all_Packages.xz deb.debian.org_debian_dists_testing_Release.gpg
deb.debian.org_debian_dists_testing_main_binary-amd64_Packages.xz deb.debian.org_debian_dists_unstable_InRelease
deb.debian.org_debian_dists_testing_main_binary-i386_Packages.xz deb.debian.org_debian_dists_unstable_Release
deb.debian.org_debian_dists_testing_main_Contents-amd64.gz deb.debian.org_debian_dists_unstable_Release.gpg
16:39 β ΰ₯ βΊ π
|
1
2
|
rrs@priyasi:/tmp/set-folder$ echo 112312312321 >> deb.debian.org_debian_dists_testing_non-free_source_Sources.xz
16:40 β ΰ₯ βΊ π
|
So in this step, we tell apt-offline to install the downloaded files. This will also include the tampered file.
The output you see below is standard and reports everything to have succeeded.
But note that the tampered file is not in the list of synced files. That file is just simply missing from the list.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
|
rrs@priyasi:/tmp/set-folder$ sudo apt-offline install .
Proceeding with installation
gpgv: Signature made Friday 07 February 2020 01:55:24 PM IST
gpgv: using RSA key 16E90B3FDF65EDE3AA7F323C04EE7237B7D453EC
gpgv: Good signature from "Debian Archive Automatic Signing Key (9/stretch) <ftpmaster@debian.org>"
gpgv: Signature made Friday 07 February 2020 01:55:43 PM IST
gpgv: using RSA key 0146DC6D4A0B2914BDED34DB648ACFD622F3D138
gpgv: Good signature from "Debian Archive Automatic Signing Key (10/buster) <ftpmaster@debian.org>"
gpgv: Signature made Friday 07 February 2020 01:56:44 PM IST
gpgv: using RSA key 16E90B3FDF65EDE3AA7F323C04EE7237B7D453EC
gpgv: Good signature from "Debian Archive Automatic Signing Key (9/stretch) <ftpmaster@debian.org>"
gpgv: Signature made Friday 07 February 2020 01:56:45 PM IST
gpgv: using RSA key 0146DC6D4A0B2914BDED34DB648ACFD622F3D138
gpgv: Good signature from "Debian Archive Automatic Signing Key (10/buster) <ftpmaster@debian.org>"
gpgv: Signature made Friday 07 February 2020 01:56:58 PM IST
gpgv: using RSA key 16E90B3FDF65EDE3AA7F323C04EE7237B7D453EC
gpgv: Good signature from "Debian Archive Automatic Signing Key (9/stretch) <ftpmaster@debian.org>"
gpgv: Signature made Friday 07 February 2020 01:56:58 PM IST
gpgv: using RSA key 0146DC6D4A0B2914BDED34DB648ACFD622F3D138
gpgv: Good signature from "Debian Archive Automatic Signing Key (10/buster) <ftpmaster@debian.org>"
deb.debian.org_debian_dists_testing_contrib_Contents-amd64.gz synced.
deb.debian.org_debian_dists_testing_contrib_Contents-i386.gz synced.
deb.debian.org_debian_dists_testing_contrib_binary-all_Packages.xz synced.
deb.debian.org_debian_dists_testing_contrib_binary-amd64_Packages.xz synced.
deb.debian.org_debian_dists_testing_contrib_binary-i386_Packages.xz synced.
deb.debian.org_debian_dists_testing_contrib_i18n_Translation-en.bz2 synced.
deb.debian.org_debian_dists_testing_contrib_source_Sources.xz synced.
deb.debian.org_debian_dists_testing_main_Contents-amd64.gz synced.
deb.debian.org_debian_dists_testing_main_Contents-i386.gz synced.
deb.debian.org_debian_dists_testing_main_binary-all_Packages.xz synced.
deb.debian.org_debian_dists_testing_main_binary-amd64_Packages.xz synced.
deb.debian.org_debian_dists_testing_main_binary-i386_Packages.xz synced.
deb.debian.org_debian_dists_testing_main_i18n_Translation-en.bz2 synced.
deb.debian.org_debian_dists_testing_main_source_Sources.xz synced.
deb.debian.org_debian_dists_testing_non-free_Contents-amd64.gz synced.
deb.debian.org_debian_dists_testing_non-free_Contents-i386.gz synced.
deb.debian.org_debian_dists_testing_non-free_binary-all_Packages.xz synced.
deb.debian.org_debian_dists_testing_non-free_binary-amd64_Packages.xz synced.
deb.debian.org_debian_dists_testing_non-free_binary-i386_Packages.xz synced.
deb.debian.org_debian_dists_testing_non-free_i18n_Translation-en.bz2 synced.
16:41 β ΰ₯ βΊ π
|
So, in the above example, apt-offline discarded the tampered file and the final exit of the command was a success.
Now, let’s run the same command with the ‘–verbose’ switch. Below is the output.
Notice the highlighted line below, where it reports that the file is tampered and does not match the checksum
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
|
rrs@priyasi:/tmp/set-folder$ sudo apt-offline install . --verbose
VERBOSE: Namespace(allow_unauthenticated=False, func=<function installer at 0x7f6a6c7c54d0>, install='.', install_simulate=False, install_src_path=None, skip_bug_reports=False, skip_changelog=False, strict_deb_check=False, verbose=True)
VERBOSE: No changelog available
Proceeding with installation
VERBOSE: {}
VERBOSE: Great!!! No bugs found for all the packages that were downloaded.
VERBOSE: APT Signature verification path is: ['/etc/apt/trusted.gpg.d/', '/etc/apt/trusted.gpg']
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg to the apt-offline keyring
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg to the apt-offline keyring
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg to the apt-offline keyring
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-stretch-automatic.gpg to the apt-offline keyring
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-stretch-security-automatic.gpg to the apt-offline keyring
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-stretch-stable.gpg to the apt-offline keyring
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-buster-automatic.gpg to the apt-offline keyring
VERBOSE: Adding /etc/apt/trusted.gpg.d/debian-archive-buster-security-automatic.gpg to the apt-offline keyring
.....snipped.....
VERBOSE: localFile ./deb.debian.org_debian_dists_testing_non-free_Contents-amd64.gz Integrity with checksum 024957d30be2acbb9e66c9802f825115d32437420300a2b28ab60ae4ecb76fcf matches
VERBOSE: localFile ./deb.debian.org_debian_dists_testing_non-free_Contents-i386.gz Integrity with checksum 5266d2f3ea41c4e988e71b4bbe58dd1178a23ce1ed50908c73a0cb39201136e3 matches
VERBOSE: localFile ./deb.debian.org_debian_dists_testing_non-free_binary-all_Packages.xz Integrity with checksum 9f0f3aa5560452d45f82c5121ea844c68e641c8fbb56ef69d570c641b6cce662 matches
VERBOSE: localFile ./deb.debian.org_debian_dists_testing_non-free_binary-amd64_Packages.xz Integrity with checksum 811f7752a13dfcbd748478dda267fb810c52fc14769d2d5c7871c75e35350d66 matches
VERBOSE: localFile ./deb.debian.org_debian_dists_testing_non-free_binary-i386_Packages.xz Integrity with checksum 7df3512b5da7258613774921023d68c71858d89fddafd694e2dfd19cef54314b matches
VERBOSE: localFile ./deb.debian.org_debian_dists_testing_non-free_i18n_Translation-en.bz2 Integrity with checksum 1bf3cd0cff6fadf1a74280912c3229374344cd6c347d2f533b001843d84b236d matches
VERBOSE: localFile ./deb.debian.org_debian_dists_testing_non-free_source_Sources.xz integrity doesn't match to checksum a94589ab3c204bb4d710d72ea21abac8007b14e5c5dacbe43be07c51ba5f0a0a
VERBOSE: Synchronized file to /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_Contents-amd64
VERBOSE: /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_Contents-amd64 file synced to APT.
deb.debian.org_debian_dists_testing_contrib_Contents-amd64.gz synced.
VERBOSE: Synchronized file to /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_Contents-i386
VERBOSE: /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_Contents-i386 file synced to APT.
deb.debian.org_debian_dists_testing_contrib_Contents-i386.gz synced.
VERBOSE: Synchronized file to /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_binary-all_Packages
VERBOSE: /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_binary-all_Packages file synced to APT.
deb.debian.org_debian_dists_testing_contrib_binary-all_Packages.xz synced.
VERBOSE: Synchronized file to /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_binary-amd64_Packages
VERBOSE: /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_binary-amd64_Packages file synced to APT.
deb.debian.org_debian_dists_testing_contrib_binary-amd64_Packages.xz synced.
VERBOSE: Synchronized file to /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_binary-i386_Packages
VERBOSE: /var/lib/apt/lists/deb.debian.org_debian_dists_testing_contrib_binary-i386_Packages file synced to APT.
deb.debian.org_debian_dists_testing_contrib_binary-i386_Packages.xz synced.
.....snipped.....
16:42 β ΰ₯ βΊ π
|
This is pretty much the validation required and done by apt-offline for apt meta Packages files.
Please do file bug reports if you think the overall exit status of apt-offline under such scenarios should be different than what it is currently.
For the tampered meta Packages files:
- should the visual representation be different ?
- Should an error be printed ?
- What about the exit status ?
Similarly, for the ‘get’ operation:
- Should we do something different for non-existing localization files on the repository server ?
- Is there any different way to go through the supported list of compression types for meta files ?
apt-offline allows a user to install a new package and all its dependencies easily on the offline machine. The below workflow will demonstrate the same and will also go through the tampering of the .deb files and see how apt-offline/apt deals with it.
In below example, a user wants to install the gnome-todo package on the offline machine, which has a couple of dependencies.
rrs@priyasi:/tmp/set-folder$ sudo apt install gnome-todo
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
The following NEW packages will be installed:
gnome-todo gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
0 upgraded, 5 newly installed, 0 to remove and 1 not upgraded.
Need to get 784 kB of archives.
After this operation, 2,337 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.
16:47 β ΰ₯ βΉ π=> 1
The below command generates a (signature) file, which will include all details about requested package and its dependencies.
1
2
3
|
rrs@priyasi:/tmp/set-folder$ sudo apt-offline set /tmp/gnome-todo.uris --install-packages gnome-todo
Gathering installation details for package ['gnome-todo']
16:48 β ΰ₯ βΊ π
|
Below is the usual step to be performed on the online machine with the generated gnome-todo.uris signature file.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
rrs@priyasi:/tmp/set-folder$ apt-offline get /tmp/gnome-todo.uris --download-dir /tmp/gnome-todo --bug-reports --threads 3
Fetching APT Data
WARNING: If you are on a slow connection, it is good to
WARNING: limit the number of threads to a low number like 2.
WARNING: Else higher number of threads executed could cause
WARNING: network congestion and timeouts.
Downloading libpeas-common - 187 KiB
Downloading libpeas-1.0-0 - 196 KiB
Downloading gnome-todo-common - 228 KiB
libpeas-common done
Fetching bug report for libpeas-common
libpeas-1.0-0 done
Fetching bug report for libpeas-1.0-0
gnome-todo-common done
Fetching bug report for gnome-todo-common
Fetched bug report for libpeas-common
Downloading libgnome-todo - 6 KiB
libgnome-todo done
Fetching bug report for libgnome-todo
Fetched bug report for gnome-todo-common
Downloading gnome-todo - 146 KiB
gnome-todo done
Fetching bug report for gnome-todo
Fetched bug report for libpeas-1.0-0
Fetched bug report for libgnome-todo
Fetched bug report for gnome-todo
5 / 5 items: [##############################] 100.0% of 765 KiB
Downloaded data to /tmp/gnome-todo
16:49 β ΰ₯ βΊ π
|
Note: The fact is that apt-offline will not do any checksum validation for the .deb files. The validation is completely delegated to apt.
rrs@priyasi:/tmp/gnome-todo$ sudo apt-offline install -h
usage: apt-offline install [-h] [--verbose] [--simulate]
[--install-src-path INSTALL_SRC_PATH]
[--skip-bug-reports] [--skip-changelog]
[--allow-unauthenticated] [--strict-deb-check]
apt-offline-download.zip | apt-offline-download/
positional arguments:
apt-offline-download.zip | apt-offline-download/
Install apt-offline data, a bundle file or a directory
optional arguments:
-h, --help show this help message and exit
--verbose Enable verbose messages
--simulate Just simulate. Very helpful when debugging
--install-src-path INSTALL_SRC_PATH
Install src packages to specified path.
--skip-bug-reports Skip the bug report check
--skip-changelog Skip display of changelog
--allow-unauthenticated
Ignore apt gpg signatures mismatch
--strict-deb-check Perform strict checksum validaton for downloaded .deb
files
16:50 β ΰ₯ βΊ π
and from the manpage:
--strict-deb-check
With this option enabled, apt-offline delegate's .deb package checksum validation to apt. While the .debs are already availβ
able, they are stored in the temporary apt cache, where apt validates its checksum, before considering it for further proβ
cessing. Note: This does have the caveat that apt may need network availability even though it doesn't download anything
over the network. But it does invoke the download routines and realizes that the payload is already available. It then furβ
ther proceeds with checksum validation
The default behavior is to not do strict checksum validation for .deb files. Instead, apt-offline copies the .deb files to
apt's download location. apt still does size validation of the available .deb files and discards them in case there is a misβ
match.
Before we proceed with the example of checksum verification for .deb files, lets do a pristine run of the downloaded files, without any tampering to them.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
|
rrs@priyasi:/tmp/gnome-todo$ sudo apt-offline install .
Proceeding with installation
Following are the list of bugs present.
822525 gnome-todo : gnome-todo: Memory leak while loading local and remote lists
853114 gnome-todo : no longer loads caldav lists
883961 libgnome-todo : libgnome-todo: Not actually a library
829470 libpeas-1.0-0 : libpeas: Python Plugin Broken
(Y) Yes. Proceed with installation
(N) No, Abort.
(R) Redisplay the list of bugs.
(Bug Number) Display the bug report from the Offline Bug Reports.
(?) Display this help message.
What would you like to do next: (y, N, ?)y
gnome-todo_3.28.1-5_amd64.deb file synced.
libgnome-todo_3.28.1-5_amd64.deb file synced.
gnome-todo-common_3.28.1-5_all.deb file synced.
libpeas-1.0-0_1.22.0-5_amd64.deb file synced.
libpeas-common_1.22.0-5_all.deb file synced.
16:51 β ΰ₯ βΊ π
rrs@priyasi:/tmp/gnome-todo$ sudo apt install gnome-todo
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
The following NEW packages will be installed:
gnome-todo gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
0 upgraded, 5 newly installed, 0 to remove and 1 not upgraded.
Need to get 0 B/784 kB of archives.
After this operation, 2,337 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.
16:51 β ΰ₯ βΉ π=> 1
|
In the above example, everything is clean and all requirements to apt are satisfied.
Here’s one more exaple, where we invoke the non-default --strict-deb-check option.
Everything remains the same, but apt gives a prompt saying that it needs to download the payload from the web. The reality is that if you just proceed with yes, nothing gets downloaded.
Note: It is not possible to explain that with a still presentation and I’m lazy to make a motion object of it.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
|
rrs@priyasi:/tmp/gnome-todo$ sudo apt-offline install . --strict-deb-check
Proceeding with installation
Following are the list of bugs present.
822525 gnome-todo : gnome-todo: Memory leak while loading local and remote lists
853114 gnome-todo : no longer loads caldav lists
883961 libgnome-todo : libgnome-todo: Not actually a library
829470 libpeas-1.0-0 : libpeas: Python Plugin Broken
(Y) Yes. Proceed with installation
(N) No, Abort.
(R) Redisplay the list of bugs.
(Bug Number) Display the bug report from the Offline Bug Reports.
(?) Display this help message.
What would you like to do next: (y, N, ?)y
gnome-todo_3.28.1-5_amd64.deb file synced.
libgnome-todo_3.28.1-5_amd64.deb file synced.
gnome-todo-common_3.28.1-5_all.deb file synced.
libpeas-1.0-0_1.22.0-5_amd64.deb file synced.
libpeas-common_1.22.0-5_all.deb file synced.
16:52 β ΰ₯ βΊ π
rrs@priyasi:/tmp/gnome-todo$ sudo apt install gnome-todo
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
The following NEW packages will be installed:
gnome-todo gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
0 upgraded, 5 newly installed, 0 to remove and 1 not upgraded.
Need to get 784 kB of archives.
After this operation, 2,337 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://deb.debian.org/debian testing/main amd64 libpeas-common all 1.22.0-5 [192 kB]
Get:2 http://deb.debian.org/debian testing/main amd64 libpeas-1.0-0 amd64 1.22.0-5 [201 kB]
Get:3 http://deb.debian.org/debian testing/main amd64 gnome-todo-common all 3.28.1-5 [234 kB]
Get:4 http://deb.debian.org/debian testing/main amd64 libgnome-todo amd64 3.28.1-5 [6,260 B]
Get:5 http://deb.debian.org/debian testing/main amd64 gnome-todo amd64 3.28.1-5 [150 kB]
Retrieving bug reports... Done
Parsing Found/Fixed information... Done
.....snipped.....
16:53 β ΰ₯ βΊ π
|
To sum it up, this one is an odd case because though nothing for the debs is downloaded, BUT, the network needs to be active for this co-routine to run. If, say, the network is unavailable, apt complains. I haven’t checked, but apt does invoke some network code.
But no payload is downloaded. apt just validates and realizes that all the to-be-downloaded data, is intact and available.
rrs@priyasi:/tmp/gnome-todo$ echo fasdfadsfasdfasdfasd >> gnome-todo_3.28.1-5_amd64.deb
16:54 β ΰ₯ βΊ π
rrs@priyasi:/tmp/gnome-todo$ sudo apt clean
16:54 β ΰ₯ βΊ π
So we tampered one of the .deb files, gnome-todo_3.28.1-5_amd64.deb. And ask apt-offline to run its ‘install’ operation along with the new --strict-deb-check option.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
rrs@priyasi:/tmp/gnome-todo$ sudo apt-offline install . --strict-deb-check
Proceeding with installation
Following are the list of bugs present.
822525 gnome-todo : gnome-todo: Memory leak while loading local and remote lists
853114 gnome-todo : no longer loads caldav lists
883961 libgnome-todo : libgnome-todo: Not actually a library
829470 libpeas-1.0-0 : libpeas: Python Plugin Broken
(Y) Yes. Proceed with installation
(N) No, Abort.
(R) Redisplay the list of bugs.
(Bug Number) Display the bug report from the Offline Bug Reports.
(?) Display this help message.
What would you like to do next: (y, N, ?)y
gnome-todo_3.28.1-5_amd64.deb file synced.
libgnome-todo_3.28.1-5_amd64.deb file synced.
gnome-todo-common_3.28.1-5_all.deb file synced.
libpeas-1.0-0_1.22.0-5_amd64.deb file synced.
libpeas-common_1.22.0-5_all.deb file synced.
16:54 β ΰ₯ βΊ π
rrs@priyasi:/tmp/gnome-todo$ sudo apt install gnome-todo
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
The following NEW packages will be installed:
gnome-todo gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
0 upgraded, 5 newly installed, 0 to remove and 1 not upgraded.
Need to get 784 kB of archives.
After this operation, 2,337 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://deb.debian.org/debian testing/main amd64 libpeas-common all 1.22.0-5 [192 kB]
Get:2 http://deb.debian.org/debian testing/main amd64 libpeas-1.0-0 amd64 1.22.0-5 [201 kB]
Get:3 http://deb.debian.org/debian testing/main amd64 gnome-todo-common all 3.28.1-5 [234 kB]
Get:4 http://deb.debian.org/debian testing/main amd64 libgnome-todo amd64 3.28.1-5 [6,260 B]
Get:5 http://deb.debian.org/debian testing/main amd64 gnome-todo amd64 3.28.1-5 [150 kB]
Fetched 150 kB in 1s (141 kB/s)
Retrieving bug reports... Done
Parsing Found/Fixed information... Done
16:55 β ΰ₯ βΉ π=> 100
|
Pay attention to the downloaded data which is only 150 KiB, for the gnome-todo package, which was tampered.
Even though apt stated that it needs to download 784 KiB of data, it actually downloaded 150 KiB only. All data was already downloaded by apt-offline but we had tampered one of the files, which resulted in it being re-downloaded.
Now, lets do one more run with the default behavior of apt-offline, i.e. without the --strict-deb-check option.
This will result in apt (internally) detecting the tampering and prompting the user that the (tampered) file needs to be downloaded
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
|
rrs@priyasi:/tmp/gnome-todo$ sudo apt-offline install .
Proceeding with installation
Following are the list of bugs present.
822525 gnome-todo : gnome-todo: Memory leak while loading local and remote lists
853114 gnome-todo : no longer loads caldav lists
883961 libgnome-todo : libgnome-todo: Not actually a library
829470 libpeas-1.0-0 : libpeas: Python Plugin Broken
(Y) Yes. Proceed with installation
(N) No, Abort.
(R) Redisplay the list of bugs.
(Bug Number) Display the bug report from the Offline Bug Reports.
(?) Display this help message.
What would you like to do next: (y, N, ?)y
gnome-todo_3.28.1-5_amd64.deb file synced.
libgnome-todo_3.28.1-5_amd64.deb file synced.
gnome-todo-common_3.28.1-5_all.deb file synced.
libpeas-1.0-0_1.22.0-5_amd64.deb file synced.
libpeas-common_1.22.0-5_all.deb file synced.
16:56 β ΰ₯ βΊ π
rrs@priyasi:/tmp/gnome-todo$ sudo apt^C
16:56 β ΰ₯ βΉ π=> 130
rrs@priyasi:/tmp/gnome-todo$ sudo apt install gnome-todo
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
The following NEW packages will be installed:
gnome-todo gnome-todo-common libgnome-todo libpeas-1.0-0 libpeas-common
0 upgraded, 5 newly installed, 0 to remove and 1 not upgraded.
Need to get 150 kB/784 kB of archives.
After this operation, 2,337 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://deb.debian.org/debian testing/main amd64 gnome-todo amd64 3.28.1-5 [150 kB]
Fetched 150 kB in 0s (448 kB/s)
Retrieving bug reports... Done
Parsing Found/Fixed information... Done
.....snipped......
16:57 β ΰ₯ βΊ π
|
Notice the highlighted line, which gives a less confusing, realistic summary of what needs to be done. In this case, apt is prompting the user that 150 KiB of data needs to be downloaded, which indeed is the case.
See also