<    April 2017    >
Su Mo Tu We Th Fr Sa  
                   1  
 2  3  4  5  6  7  8  
 9 10 11 12 13 14 15  
16 17 18 19 20 21 22  
23 24 25 26 27 28 29  
30
00:10 <^7heo> ncopa: the thing now builds, but it does not pass the tests. I'll review tomorrow. I hope I can solve it, but again, if you can help; it'd be awesome
00:10 <^7heo> good night all.
00:56 Emperor_Earth joined
01:44 xsteadfastx joined
01:47 MH0815 joined
01:47 s33se joined
02:40 czart__ joined
02:42 s33se joined
03:06 blueness joined
05:05 fabled joined
05:21 <TemptorSent> 'evening fabled.
05:22 <fabled> hi
05:22 <TemptorSent> So I've been hunting high and low, but I can't find ANY way of getting the checksums in the pax headers of the apks out without horribly hackary.
05:23 <TemptorSent> Are they actually implemented beyond being stored, or am I blind?
05:23 <fabled> apk-tools reads, stores and verifies them in 'audit'
05:24 <TemptorSent> Huh, the ONLY place I saw it actually read was when it hits a conflict with a protected file.
05:25 <fabled> that's another place which checks it
05:25 <TemptorSent> APK-TOOLS.checksum.SHA1 header.
05:25 <fabled> lbu uses 'apk audit' to generate the changed file list
05:26 <TemptorSent> I was trying to analyze a codepath to use to add a list-apk-contents-with-checksums feature.
05:27 <TemptorSent> apk_archive_entry_extract isn't called anywhere but on conflict that I could find, and that's the only mention of the header fingerprint I can find in apk tools, so if you can point me at where to look so I can throw away my horrible, horrible awk script and do it right, I'd much appreciate it :)
05:27 <fabled> archive_entry_extract is called for every file when an .apk is being extracted
05:28 <TemptorSent> So the suffix is only used on conflict?
05:28 <fabled> yes
05:28 <TemptorSent> Wow, that's a hard code path to follow!
05:30 <TemptorSent> Okay, so the short of it appears there is no easy way to simply itterate over the members of the archvie and spit out the values without setting up an entire context and extracting.
05:31 <TemptorSent> So all file access essentially passes through the database both ways?
05:31 <fabled> pretty much yes
05:31 <TemptorSent> Or at least that structure...
05:32 <TemptorSent> Okay, that's beyond a quick few lines it's looking like :/
05:33 <TemptorSent> I'll have to see if my awk script can put up with ugly input, absent ANY posix pax compliant tool.
05:34 <TemptorSent> I couldn't find anything that could list the headers, but at least 7zip would dump them to directories.
05:35 <TemptorSent> That's a pretty sad state of affairs for a supposedly standard archive format!
05:35 <TemptorSent> Anyway, that aside, mkinitfs in my branch is refactored and supports everything the original did.
05:36 <TemptorSent> I'm working on cleaning up the kernel updating code to actually make sense :)
05:36 <fabled> ok, do note that mkinitfs is packaged separately because it is used also to construct hard-disk install system's initramfs when linux kernel is updated
05:37 <TemptorSent> Right, I have it to the point it should handle that or only need minor adjustments to handle that.
05:37 <TemptorSent> It is almost exactly compatible with the existing script, including all feature names (in a compat direcotry)
05:39 <TemptorSent> It should drop in unless someone has done custom features, in which case there's a converter script there too :)
05:40 <TemptorSent> I'm going to suggest using the directory /usr/share/mkalpine for all of the related scripts, including mkinitfs, mkimage, update-kernel, and to be added mkmodloop.
05:42 <TemptorSent> That would keep /etc used for user configs, not package details.
05:43 <TemptorSent> I intend to transition the format of the files/modules entries themselves to be prefixed by the source package names so we can quickly discover what files come from what apks.
05:44 <TemptorSent> And make very clear anything that comes from the filesystem.
05:45 <TemptorSent> Also, the modules don't need paths specified in most cases to be discovered, so I'm going to use the modinfo output to get the filenames, which should be cannonical even if an alias is given for the module name.
05:46 <TemptorSent> Let's see how long before iRCFrEAK hits the K-Line...
05:49 <TemptorSent> Right at the moment, I'm working on changing update kernel to use the following pieces broken out into their own tools: stagekernel, which takes a kernel build directory or package name and a list of other build directories and apks to process.
05:51 <TemptorSent> It installs/unpacks each to it's own directory, creates a manifest file, then links all files in all extracted/installed directories into a merged-root.
05:52 <TemptorSent> Then it creates a merged manifest file which indicates the package name, package version, checksum, and path for each file.
05:54 <TemptorSent> It then runs depmod and uses the manifest to make a list of package deps for modules.
05:55 IRCFrEAK joined
05:56 <TemptorSent> Once staged, mkmodloop and mkinitfs can use the manifest and copy/extract only what they need, while still being able to trace the origins of each file they contain to an original signed apk.
05:56 ostera joined
05:56 IRCFrEAK left
05:57 <TemptorSent> Make sense fabled?
05:59 <TemptorSent> The modloop and mkinitfs module handling could be merged to the point of specifying features for either in same fashion using the same definitions.
06:01 <TemptorSent> That would allow initfs to only have what's needed for boot, while making a modloop that contains only what the user needs on their system and can be stored in ram, leaving the boot device able to be removed.
06:03 <TemptorSent> Until that's done, using the existing kitchen sink is easy enough :)
06:05 <TemptorSent> It will have the added bonus that the image generation will only need to stage a given kernel once.
06:08 <TemptorSent> update-kernel itself becomes a matter of calling stagekernel and copying the required files to their destination, preferably with a manifest of everything install extracted and installed.
06:24 cyteen joined
06:32 ostera-mobile joined
06:37 ostera-mobile joined
06:41 ostera joined
06:54 ostera joined
07:02 blueness joined
07:05 schndr_ joined
07:08 ostera joined
07:19 tru_tru joined
07:21 ostera joined
07:22 stwa joined
07:25 ostera-mobile joined
07:35 ostera1 joined
07:48 ostera1 joined
08:02 ostera1 joined
08:09 cyteen joined
08:16 ostera1 joined
08:21 cyteen joined
08:29 ostera1 joined
08:43 royger joined
08:43 ostera1 joined
08:43 schndr_ joined
08:56 ostera2 joined
09:04 cyteen_ joined
09:10 ostera2 joined
09:23 ostera2 joined
09:37 ostera2 joined
09:39 ncopa joined
09:39 ncopa joined
09:50 <^7heo> ncopa: ping
09:51 ostera2 joined
10:02 <ncopa> ^7heo: pong
10:02 <ncopa> hi
10:04 ostera2 joined
10:06 <^7heo> ncopa: hey man
10:06 <^7heo> ncopa: so I tried to fix that mkinitfs deported header feature.
10:06 <^7heo> Good thing is that I got the rebase through.
10:06 <^7heo> Finally.
10:07 <ncopa> good
10:07 <^7heo> I'm not very comfortable with rebasing so much code I don't know over so much code I don't know.
10:07 <^7heo> but I did it, and I hope I got most of the logic right
10:07 <^7heo> it didn't compile when I finished the rebase, but that's because I blindly copy-pasted a function call around while the function signature had changed in the meanwhiel.
10:07 <^7heo> s/whiel/while/
10:08 <^7heo> Now it does compile, but it does not work as intended, it does not pass the tests.
10:08 <^7heo> So I am working on finding what is wrong now.
10:08 <^7heo> I'm on 2e3a4f61677f2ab4c65a4f128ec57d1fce87599d
10:08 <^7heo> And the test is not working anymore, already.
10:09 <^7heo> I guess that if I fix that, I could probably easily get the rest fine.
10:10 <^7heo> The thing is, as I read the patch, I don't see anything blatantly wrong: https://github.com/alpinelinux/mkinitfs/pull/11/commits/2e3a4f61677f2ab4c65a4f128ec57d1fce87599d
10:12 <^7heo> ncopa: when I run the test, I get a crapload of `mdev: unknown user/group 'root:input' on line 8x` with x being in 5..9
10:12 <^7heo> ncopa: Any idea where that could be coming from?
10:12 <ncopa> do you have a group named "input"?
10:12 <ncopa> i think its something in your /etc/mdev.conf
10:13 <^7heo> no. I do not.
10:13 <^7heo> Should I?
10:13 <^7heo> I mean my computer works as intended, so...
10:13 <ncopa> i think you should have a group named "input"
10:14 <^7heo> ohmydog I don't like files which start with '# This is a sample ...'
10:14 <^7heo> ncopa: right, under '# input stuff'
10:14 fekepp joined
10:14 <^7heo> I take it is pretty new, was I never saw such a message ever before.
10:15 <ncopa> i dont think its new
10:16 <^7heo> Well, my current running install is from 3.2 or something
10:16 <^7heo> and I have upgraded it every other month (or couple thereof)
10:17 <^7heo> but I never had anything happening with that.
10:17 <^7heo> ok created.
10:18 ostera2 joined
10:18 <^7heo> I'm gonna re-run the test now.
10:18 <^7heo> To see if I get a different input.
10:18 <^7heo> Much different.
10:19 <^7heo> Thanks ncopa, problem half solved ;]
10:31 ostera2 joined
10:32 blueness joined
10:33 <^7heo> ncopa: yeah so it has to do with the way the whole nlplug-findfs has been changed right after I submitted my patchset.
10:33 <^7heo> ncopa: the existing variables are not holding the same data anymore.
10:35 <xentec> I see the g++ package contains C headers for java. shouldn't they be in gcc-java? https://pkgs.alpinelinux.org/contents?page=11&arch=x86_64&branch=edge&name=g%2B%2B
10:45 ostera2 joined
10:52 <nmeum> the APKBUILD examples in the abuild repository and the APKBUILDs created by newapkbuild still use explicity `|| return 1` statements. Shouldn't those be removed since abuild runs with `set -e` nowadays?
10:59 ostera2 joined
11:00 <ashb> Is it in a function?
11:01 <ashb> yeah, I think that's still needed. Cos `-e` behaves differently in functions
11:01 <ashb> I never remember exactly how it behaves in a function, but I know it does behave differently
11:12 ostera2 joined
11:19 <pickfire> What should I do with http://patchwork.alpinelinux.org/patch/3199/?
11:20 <nmeum> ashb: actually it should only behave differently in a multi-command pipeline, when executing the compound list following the while, until, if, or elif reserved word, a pipeline beginning with the ! reserved word, or any command of an AND-OR list other than the last
11:22 <nmeum> in functions it works as expected so imho we don't need the explicit `|| return 1` in the templates
11:22 <ashb> All I can remember is that it I've been stung by it in functions :D
11:26 ostera2 joined
11:29 ginjachris joined
11:29 <ginjachris> hello all
11:31 <clandmeter> anyone has knowledge about acme challenges?
11:39 ostera2 joined
11:45 LebedevRI joined
11:45 <LebedevRI> _ikke_: hi. 2.2.4
11:53 ostera2 joined
11:58 <^7heo> ncopa: currently it seems that the test script has an issue.
11:58 <ncopa> ^7heo: not surprised
11:59 <^7heo> I mean I had to fix a couple of things along the way
12:00 blueness joined
12:00 <^7heo> but yeah right now the problem I have is in the testing script.
12:00 <^7heo> ncopa: and why aren't you surprised?
12:03 <^7heo> Now I have a better test script, it doesn't crash on the normal testing anymore, but it crashes on the header test.
12:03 <^7heo> So there's progress.
12:05 rdutra joined
12:05 ferseiti joined
12:06 ostera2 joined
12:08 vakartel joined
12:09 schndr_ joined
12:16 leitao joined
12:16 <ashb> clandmeter: a little bit
12:18 gromero joined
12:20 ostera2 joined
12:23 blueness joined
12:24 <ashb> /bin/sh -c 'set -e; foo() { false; echo here?; }; foo && echo "foo passed"; echo $?'
12:24 <ashb> nmeum: ^^ that's when set -e in functions behaves oddly. If the function itself is used in a compound/conditional
12:24 <ashb> That will show "here?"
12:25 <ashb> (Though if its the last statement in a function it's oka
12:26 blueness joined
12:31 farosas joined
12:34 ostera2 joined
12:36 medber joined
12:47 ostera2 joined
12:51 <_ikke_> LebedevRI: Thanks
12:52 <LebedevRI> _ikke_: i see that .3 got skipped, and no one complained? :/
12:52 <_ikke_> LebedevRI: Nope, no one did. (I did see it was released, but forgot to push the updated package)
12:53 <LebedevRI> i guess no one is using it on alpine then
12:54 <_ikke_> I only ever heard about one person wanting it on Alpine, but never heard of them again
12:55 <LebedevRI> i did bump the thread on the maillist, maybe someone will respond
13:01 ostera2 joined
13:10 tkharju joined
13:14 ostera2 joined
13:28 ostera2 joined
13:38 mosez joined
13:41 ostera2 joined
13:55 ostera2 joined
14:02 <^7heo> ncopa: I think I fixed it.
14:03 <^7heo> ncopa: please review in depth if you can; but it passes the tests locally.
14:05 <^7heo> AND IN GITHUB TOO!!!! \o/
14:06 <algitbot> \o/
14:06 blueness joined
14:09 ostera2 joined
14:22 Meths joined
14:22 ostera2 joined
14:28 ncopa joined
14:28 ncopa joined
14:29 Meths joined
14:35 Meths joined
14:35 ncopa joined
14:35 ncopa joined
14:36 ostera2 joined
14:39 stwa joined
14:49 ostera2 joined
14:56 Meths joined
15:03 ostera2 joined
15:06 Meths joined
15:09 ncopa joined
15:09 ncopa joined
15:10 faffolter joined
15:10 faffolter joined
15:10 <^7heo> Klowner: ping
15:13 ferseiti joined
15:13 Meths joined
15:16 ncopa joined
15:16 ncopa joined
15:17 ostera2 joined
15:24 Meths joined
15:25 LouisA joined
15:25 <TemptorSent> 'morning ncopa - how's it going?
15:26 <ncopa> hi
15:26 <ncopa> trying to figure out why by bluetooth controller is not working
15:27 <TemptorSent> I don't know if you caught up on the backlog, but my mkinitfs script should be able drop in as a replacement.
15:27 <TemptorSent> Hmm, failing to initilize or failing to connect?
15:28 tmh1999 joined
15:28 <ncopa> [ 5.816148] Bluetooth: hci0 command 0xfc05 tx timeout
15:28 <ncopa> [ 5.816155] Bluetooth: hci0: Reading Intel version information failed (-110)
15:28 <ncopa> looks like kernel or firmware issue
15:28 <ncopa> but reverting to older kernel didnt solve it
15:29 <ncopa> neither did downgrading the firmware
15:29 <ncopa> so i dunno whats wrong
15:29 <TemptorSent> Yeah, that is a bit odd.. It's a usb->BT adapter?
15:29 <ncopa> TemptorSent: do you have a pull request for mkinitfs git repo?
15:29 <ncopa> yes, its on the usb bus
15:29 <ncopa> but on the motherboard
15:30 <TemptorSent> ncopa: The first thing I would suspect is something enumerating wrong and it not initilizing the endpoints fully.
15:30 ostera2 joined
15:31 minimalism joined
15:32 <TemptorSent> ncopa: No PR to the mkinitfs repo as of yet, I'm not sure if that's the best way of going about integrating it either...
15:34 <ncopa> so you prefer that we maintain 2 different mkinitfs?
15:34 <TemptorSent> ncopa: I could try to merge it into the existing repo, but it would probably be more confusing than not.
15:35 <TemptorSent> ncopa: No, ideally we could transition the existing mkinitfs repo to just nlplug.
15:36 <TemptorSent> ncopa: And provide mkinitfs with 'mkalpine' or whatever the package name would be that includes the functionality of mkinitfs, update-kerenl, etc.
15:36 <^7heo> TemptorSent: aside from the revamp, what did you fix with your version?
15:37 <TemptorSent> ^7heo: As far as mkinitfs itself, I refactored or rewrote the entire thing :)
15:38 <^7heo> yes, and aside from that revamp, what did you fix?
15:38 <TemptorSent> ^7heo: I'm not messing with nlplug at all at this point, and my proposed init changes are not integrated.
15:38 <^7heo> yeah I know ;)
15:38 <^7heo> I'm asking what new features you implemented.
15:38 <^7heo> if any.
15:38 <TemptorSent> ^7heo: File structure for features was the main goal.
15:39 <^7heo> What is a File structure for features?
15:39 <TemptorSent> ^7heo: So the feature definitions can be defined in a modular manner.
15:39 <^7heo> Ah
15:39 <^7heo> Gotchat.
15:39 <^7heo> s/chat/cha/
15:39 <TemptorSent> ^7heo: Look at /etc/mkinitfs/features.d for the current state.
15:40 <^7heo> TemptorSent: yeah that's a list of modules to load, or?
15:40 <TemptorSent> ^7heo: Adding the 'ata' feature ends up including every ata-like driver known to man in the initfs.
15:40 <TemptorSent> ^7heo: files/modules to be included in the initfs.
15:41 <^7heo> ah yeah
15:41 <^7heo> And your goal is to reduce the size of the initfs?
15:42 <TemptorSent> ^7heo: So with them being stored in shell scripts instead of flat files, we can do things like include several feature sets in one file and have them inherit where desired.
15:42 <TemptorSent> ^7heo: Yeah, removing everthing we don't need in an initfs :)
15:44 ostera2 joined
15:44 <TemptorSent> ^7heo: I'm just getting ready to add the packages as tags to the files included in a feature, so 'zfs:/usr/sbin/zpool' for instance.
15:45 <TemptorSent> ^7heo: And probably going through to change the kernel flavor handling in mkimage at the same time so everything is consistent.
15:47 <TemptorSent> ^7heo: With the intent of doing something like 'zfs-KFLAVOR:zfs' for the modules and it just working.
15:49 <TemptorSent> So part of the problem with splitting it out and packaging it in place of the original mkinitfs script is that the logic for handling things like kernel versions/flavors/etc would have to be duplicated for each tool that needs them.
15:51 Meths joined
15:54 <TemptorSent> I'm working on replacing the update-kernel logic entirely with a seperate stagekernel tool, a mkmodloop tool, and a very simple installer.
15:57 ostera2 joined
15:59 <TemptorSent> This gives us the ability to make modloops that are customized in their contents using the same type of definitions we're using for mkinitfs.
16:10 Meths joined
16:11 ostera2 joined
16:16 Meths joined
16:21 Meths joined
16:25 ostera2 joined
16:27 Meths joined
16:38 ostera2 joined
16:41 Meths joined
16:48 tmh1999 joined
16:52 ostera2 joined
17:01 tkharju joined
17:05 Meths joined
17:05 ostera2 joined
17:11 cyteen joined
17:15 Meths joined
17:19 ostera2 joined
17:20 Meths joined
17:27 Meths joined
17:32 ostera2 joined
17:39 Meths joined
17:46 ostera2 joined
17:49 <TemptorSent> Query: For custom kernel / module builds, how should we identify the version?
17:50 Meths joined
18:00 ostera2 joined
18:11 Meths joined
18:13 ostera2 joined
18:21 <TemptorSent> Would $pkgname-$pkgver@$abirelseae be appropriate?
18:23 <TemptorSent> Giving for instance 'zfs-0.6.5.9-r0@4.9.19-0-grsec' if one were to compile zfs themselves
18:23 <kaniini> apk already uses @ for pinning to a specific repo
18:25 <TemptorSent> kaniini: Okay, I'm not sure if that actually would conflict with the use for custom builds, but I'm open to suggestions for how to handle them :)
18:27 ostera2 joined
18:28 <TemptorSent> kaniini: Would '=' or '%' be preferable?
18:28 <kaniini> TemptorSent: @ is not an allowed character in package atoms
18:29 <TemptorSent> kaniini: Right, but these would not be packages per-se, since they're just build directories.
18:30 <TemptorSent> But scrap the '@' sign if it would cause confusion.
18:30 <kaniini> i think it would be preferable to avoid using @ anyway, as it would be confusing
18:32 <TemptorSent> So, '=','%','^', or '~' are options, or something more complex if we really need it.
18:33 faffolter joined
18:34 <TemptorSent> kaniini: The use is for staging kernel related files from custom builds for use by mkinitfs, modloop, etc.
18:34 kaiyou left
18:34 Kug3lis joined
18:35 <TemptorSent> And creating a manifest to identify where each file came from along with checksums for it.
18:36 <TemptorSent> Eventually, it could also handle the building of such out of tree objects automatically as well.
18:36 <TemptorSent> And build all of your custom modules/firmware for you when you update your kernel.
18:37 <TemptorSent> So that's the need for the versioning...
18:41 ostera2 joined
18:41 Ganwell joined
18:42 <TemptorSent> You'd pass it something like 'make_modules_install:/usr/src/mypkg-3.1.5^4.9.19-0-grsec' and it will change to /usr/src/mypkg-3.1.5 and run make modules_install to an output directory of $staging/4.9.19-0-grsec/mypkg-3.1.5/modules
18:42 blueness joined
18:44 <TemptorSent> And in the manifest would have an entry 'custom_modules:mypkg-3.1.5^4.9.19-0-grsec SHA512:<checksum> <file>' for each file installed in that directory (as well as sha256 and sha1 currently)
18:48 <TemptorSent> But it's needed to tell the difference between entries for mypkg-3.1.5 built for 4.9.11-1-grsec vs 4.9.19-0-grsec
18:49 <TemptorSent> kaniini: If you have thoughts on a cleaner way to handle that, I'm all ears.
18:51 <TemptorSent> It's also needed to allow building multiple versions of a package against the same kernel (which seems broken in apk right now)
18:53 <TemptorSent> In fact, the whole kernel flavor handling is a bit kludgy IMHO, with module packages created for each flavor they want to support rather than just being compiled when the kernel update is.
18:54 <TemptorSent> The module packages should be able to automatically detect which kernels are installed (install-if?) and install modules for all of them each update.
18:55 <TemptorSent> I'll fix the rest of that later, right now I just want to nail down a consistent means of specifying it.
18:56 Meths joined
18:59 <TemptorSent> The other possible gotcha is what do we do about custom kernels with the same versions as dist? Do we default to naming them -custom? -$builddate? -$hostname?
19:00 <tdtrask> any easy way to split an apk filename into package name and version?
19:00 <tdtrask> ie. mini_httpd-1.23-r0.apk _> mini_httpd and 1.23-r0
19:01 <* tdtrask> is sure it's done all the time in apk
19:01 <TemptorSent> tdtrask: several options, depending on the context
19:01 <tdtrask> shell script
19:02 <skarnet> there are ways.
19:02 <TemptorSent> In a shell script, trimming back to the last -followed by a version number works.
19:02 <skarnet> there's no *easy* way.
19:02 <tdtrask> specifically, need to write a shell script to cleanse a local repo that contains multiple version of the same packages
19:02 <tdtrask> and only keep the most recent
19:03 <kaniini> why do we need to care about custom builds
19:03 <kaniini> they should use aports
19:03 <TemptorSent> What I do is hack off .apk, hack off -r*, hack of -[0-9._]*
19:03 <tdtrask> TemptorSent: as seen in the example above, the last '-' is part of the version
19:03 <tdtrask> ah, three steps
19:04 <TemptorSent> tdtrask: Yeah, that way you capture each piece of info as you go too.
19:04 <tdtrask> are we guaranteed to see the '-' characters in that manner?
19:04 <tdtrask> if so, it's easy enough to parse
19:04 <TemptorSent> kaniini: Because MANY people custom build their own kernels or need to use some modules that require an out of tree build.
19:04 <kaniini> so use aports
19:05 <TemptorSent> tdtrask: I believe so.
19:05 <TemptorSent> kaniini: Not realistic.
19:06 <tdtrask> TemptorSent: thanks
19:06 Meths joined
19:06 <TemptorSent> kaniini: At least not without making aports usable without installing all of aports :)
19:07 <TemptorSent> tdtrask: I haven't verified there are no outliers, but that's what I gathered from the spec for version numbers.
19:10 <TemptorSent> tdtrask: Use single '%' to do the hacking, then capture by hacking what's left off what you started with.
19:11 Meths joined
19:12 <TemptorSent> kaniini: This needs to work in the same environment update-kernel works currently. I'll be gut-shot, then drawn-and-quartered if I suggest installing aports to let the user update their kernel.
19:13 <skarnet> you'll be drawn and quartered either way. You need to decide whom you fear more.
19:13 <TemptorSent> The mob, methinks :)
19:14 <TemptorSent> And I'm trying to make everything that works currently continue to work while adding some useful information.
19:14 <kaniini> i will be in the mob if you break my stuff
19:15 <kaniini> just sayin'
19:15 <TemptorSent> kaniini: What would I be breaking?
19:15 <skarnet> "if you breaka my tests I breaka you face"
19:15 <Shiz> the more i read about this mkinitfs rewrite stuff the more i fear for its complexity
19:15 <TemptorSent> That's what I'm trying to figure out BEFORE it bites anyone.
19:16 <TemptorSent> Shiz: This is actually nothing to do with mkinitfs than providing it the files it needs.
19:17 <Shiz> doesn't change stuff
19:17 <TemptorSent> Shiz: It's providing a single tool to stage kernel related files.
19:17 <TemptorSent> Shiz: Currently, what we have is broken in many use cases.
19:18 <TemptorSent> Shiz: It only works for a specific set of configurations without major intervention.
19:18 <Shiz> and you happen to be the only person who's ran into a different configuration?
19:18 <Shiz> :p
19:18 <TemptorSent> Shiz: And breaks entirely sometimes (like it did on my system, installing modules that didn't match the kernel version)
19:19 <TemptorSent> Shiz: Have you been reading the questions here and in -linux? Dozens of people trying to work around issue.s
19:20 <TemptorSent> Shiz: It currently does not even get the dep tree right for squashfs!
19:21 <kaniini> Shiz: exactly
19:21 <TemptorSent> skarnet: Is there anything I'm running up agasint that is likely to break any existing configuration or test?
19:22 <TemptorSent> kaniini, Shiz, skarnet: Are you guys saying that the way the kernel and module building is handled now is GOOD?
19:22 <skarnet> TemptorSent: don't ask me, I'm the comic relief.
19:23 <kaniini> i am just saying
19:23 <Shiz> i'm saying i sure have never ran into issues or know anyone who has w.r.t. modules
19:23 <Shiz> :p
19:23 <kaniini> please do not break my stuff
19:23 <TemptorSent> skarnet: I value your opinion.
19:23 <TemptorSent> kaniini: I'm asking what I'm going to break by making the proposed changes!
19:24 <TemptorSent> kaniini: AFAIK, it shouldn't change ANYTHING.
19:24 <skarnet> I appreciate that, but atm I'm very unable and unwilling to focus on your issue.
19:25 <skarnet> You need to realize that your efforts are appreciated but are also a human resource hog.
19:25 <TemptorSent> skarnet: No problem, just please speak up if anything sounds like it's going to be a breaker.
19:25 <skarnet> No, you can't expect me or anyone to be the alarm bell for you.
19:26 <kaniini> when i read about things like cpio appends
19:26 <kaniini> and blah blah
19:26 <skarnet> You need to pace yourself so people can review what you're doing at their own rhythm.
19:26 <kaniini> i think "please do not break my stuff"
19:26 <TemptorSent> skarnet: Unfortunately, I don't have the years with alpine to know about the idiosyncracies.
19:27 <TemptorSent> kaniini: I'm actively trying NOT to break your stuff.
19:28 <TemptorSent> And I'm generally trying to make sure I don't do anything that would cause any breakage for users upgrading unless they've already done something out of the ordinary, in which case I've given them tools to convert as well.
19:29 <TemptorSent> kaniini: So telling people to use aports for their custom builds would be a regression compared to current functionality.
19:30 <kaniini> TemptorSent: alpine is not slackware
19:30 <kaniini> TemptorSent: you are really meant to use apk for everything
19:30 <TemptorSent> skarnet: And the only reason I'm trying to get this done on a short timescale is ncopa requested it be done for 3.6.
19:30 <Shiz> but aports is a sole alpine repo
19:30 <Shiz> unless you mean 'use abuild', in which case yes
19:31 <kaniini> yes, use abuild
19:31 <TemptorSent> kaniini: Okay, you sell ncopa and fabled on that, and I'll scrap the custom build handling they setup.
19:31 <kaniini> TemptorSent: if you replace the kernel with your own outside of our packaging, then we will assume you are knowledgeable enough to deal with the consequences of that
19:32 <kaniini> sigh
19:32 <kaniini> TemptorSent: yes, obviously the tools support custom builds
19:32 <TemptorSent> All I was asking is how to notated the kernel version for external builds so we can keep track of which files came from where and checksum them.
19:33 <kaniini> at any rate
19:33 <TemptorSent> As it stands, nothing on the system ever checksums them, including apk audit.
19:33 <kaniini> please do not break my stuff
19:33 <kaniini> yes
19:34 <kaniini> so
19:34 <kaniini> maybe there is a theme here
19:34 <kaniini> that we do not care about the integrity of said custom builds
19:34 <kaniini> because we assume you are smart enough to deal with it yourself
19:34 <TemptorSent> Unchecksummed, untracked files?
19:34 <kaniini> like i said, you're really supposed to use apk to manage your custom builds
19:34 <kaniini> maybe what you should do
19:34 <kaniini> is make a tool
19:34 <kaniini> to enable apk to track those builds more easily
19:35 <TemptorSent> That's what I'm doing!
19:35 <kaniini> then i dont udnerstand what all of this business is about
19:35 <TemptorSent> I'm making a checksummed manifest,.
19:35 <kaniini> all you really need to do is just generate an abuild
19:35 <kaniini> and then run abuild on it
19:35 <kaniini> er, apkbuild*
19:36 <kaniini> i could seriously code this in like 10 minutes
19:36 <TemptorSent> That requires installing the whole abuild framework.
19:36 <skarnet> that's what a premature optimizator would say
19:37 <TemptorSent> kaniini: Okay, clearly my needs are not the priorities of Alpine - I get that.
19:37 <TemptorSent> kaniini: What I don't get is why supporting manifests of inidivual files is so counterintuitive.
19:38 laj joined
19:39 <kaniini> oh gnoes
19:39 <kaniini> not installing abuild
19:39 <TemptorSent> kaniini: I'm trying to get the initfs to the point it can be a set of apks you install, but that still leaves dozens of files orphaned.
19:39 <* kaniini> walks away from this conversation
19:40 <TemptorSent> Look, if my work has no value, just tell me and I'll give it up as a bad job.
19:41 <TemptorSent> I really didn't intend to get this far into it in the first place, and I can just as easily purge the whole mess and get away from the computer again.
19:41 leo-unglaub joined
19:42 <kaniini> why is it an either/or thing
19:42 blueness joined
19:42 <kaniini> it's either great or i am going to go raise goats
19:42 <skarnet> that's right! #whynotboth
19:43 <skarnet> my work is great *and* I'm going to raise goats anyway
19:43 <skarnet> just because
19:43 <TemptorSent> kaniini: Because I have other things I could be spending my time on if I'm just spinning my wheels here and you could fix the problem in a few minutes rather than me taking a few days.
19:44 <TemptorSent> kaniini: And none of those other things involve a computer, and the weather is now getting nice, so I don't have much reason to waste time if that's all I'm doing.
19:44 <kaniini> as i said, why not just leverage abuild for this
19:44 <kaniini> if you are installing your own kernel, would you not want the package manager to properly be aware of it?
19:45 <TemptorSent> kaniini: I could have solved my original problem with a drive out to Utah in less time than I've spent trying to make Alpine fit for my needs.
19:45 <kaniini> gentoo have things like genkernel
19:45 <kaniini> which basically do this
19:45 <TemptorSent> kaniini: Genkernel doesn't work for shit to make a CUSTOM kernel.
19:45 <kaniini> what is a custom kernel then?
19:45 <TemptorSent> kaniini: I ALWAYS built my own kernel in gentoo/funtoo.
19:46 <skarnet> I always built my own kernel no matter what distro I was using, and I hated it when distros tried to force me to use their own shitty framework.
19:46 <TemptorSent> kaniini: One which is patched/has custom config settings/signed/etc.
19:46 <skarnet> Just for reference.
19:47 tmh1999 joined
19:47 <TemptorSent> skarnet: Agreed. Thats why I'm trying to take the resulting files and do something sane with them, not modify the build process.
19:47 <kaniini> skarnet: well there is nothing wrong with not using a framework, but you shouldnt expect tools dependent on that framework anyway to work with your custom kernel
19:47 <kaniini> if you are that deep in, you know what you're doing
19:47 <skarnet> kaniini: exactly
19:47 <kaniini> and probably aren't using an initramfs anyway
19:48 <Shiz> i think i'm fine with the mkinitfs thing not handling non-abuild/apk files
19:48 <Shiz> personally.
19:48 <skarnet> yeah, once I understood initramfs was useless in my simple case (rootfs on a good old ext2 hda), I rebuilt the kernel specifically to get rid of it. :P
19:48 <Shiz> i don't think it's an unreasonable requirement to do your kernel via abuild if you really want to
19:48 <TemptorSent> kaniini: That's going a bit far -- Often the reason you're building a custom kernel is to support just the drivers you need to load the payload into the initramfs for run-from-ram nodes.
19:48 blueness joined
19:48 <Shiz> support that
19:48 <Shiz> but
19:49 <Shiz> TemptorSent: so here's my problem
19:49 <kaniini> [citation needed]
19:49 <Shiz> when i'm building a custom kernel, i don't include module support in the first place
19:49 <Shiz> since i already can include everything i need as =y
19:49 <Shiz> so... how does a custom kernel affect the initramfs at all?
19:49 <TemptorSent> Shiz: In that case, you wouldn't be installing any modules in the initramfs.
19:49 <kaniini> if you want to use alpine tools with a custom built kernel, it should be built the alpine way
19:50 <kaniini> if you do not care about that, then don't use alpine tools
19:50 <Shiz> kaniini: i think it's even fine to use the tools with it
19:50 <Shiz> that's what the current mkinitfs does
19:50 <TemptorSent> kaniini: How does 'make menuconfig' work with abuild?
19:50 <Shiz> but going out of its way to support it... yeah
19:50 <kaniini> sure, but if he wants to use apks for the initramfs or whatever it is he is doing
19:51 <Shiz> essentially my proposition would be, for this situation:
19:51 <Shiz> use .apks and get all the checksum/validation stuff you're working on
19:51 <TemptorSent> kaniini: What it supports now is changing to the build directory and running make install (or zinstall) , make modules_install, make firmware_install, and make dtbs_install.
19:51 <Shiz> use folders and don't
19:51 <TemptorSent> Shiz: At the moment, apks actually don't make life easy to verify any of the kernel files.
19:52 <TemptorSent> Shiz: You can't verify the checksums because they're not installed by apk and thus not in the database.
19:52 <kaniini> anyway, abuild isn't really that heavy of a dependency.
19:52 <Shiz> well... why are they not installed by apk
19:52 <TemptorSent> Shiz: And extracting the checksums from apks is much more difficult than one might hope.
19:53 <TemptorSent> Shiz: Because they get staged, then coppied.
19:53 <kaniini> what shiz is saying is
19:53 <kaniini> why not just use apk
19:53 <kaniini> to unpack the apks
19:53 <kaniini> thus making them installed
19:53 <kaniini> into your initramfs image
19:53 <Shiz> yes ^
19:53 <TemptorSent> kaniini: Because currently, we CAN'T
19:53 <Shiz> apk --root <somedir>
19:53 <Shiz> it exists
19:54 <kaniini> do i really have to code an initramfs generator to demonstrate this
19:54 <Shiz> TemptorSent: why not?
19:54 <TemptorSent> Shiz: Yeah, too bad that doesn't work when you're trying to verify a file installed elsewhere.
19:54 <Shiz> just trying to understand the situation
19:55 <Shiz> elsewhere?
19:55 <TemptorSent> Shiz: The initramfs is made up of files that are pulled from whatever base directory is specified, with no idea what APKs they may relate to.
19:55 <kaniini> yes
19:55 <kaniini> what we are saying is
19:55 <Shiz> right, so why don't we change that
19:55 <kaniini> why don't you just generate some APKs
19:55 <kaniini> and embed them
19:55 <kaniini> into the initramfs
19:55 <Shiz> e.g., very simply put
19:55 <kaniini> and on top of that
19:56 <Shiz> why couldn't we re-architecture mkinitfs to take an (eventual) list of apk package names and folders
19:56 <TemptorSent> kaniini: That's what my whole proposal regarding init was!
19:56 <kaniini> we could you know, generate some APKs as part of the archive itself
19:56 <Shiz> i don't even think we need to generate new apks
19:56 <kaniini> then what the hell is all of this highly complex stuff
19:56 <Shiz> for that
19:56 <Shiz> i mean
19:56 <kaniini> Shiz: to slim them down
19:56 <Shiz> we could even extend the current mkinitfs
19:56 <kaniini> Shiz: like udeb vs deb
19:56 <Shiz> to take a new kind of features.d entry called .packages
19:56 <Shiz> e.g. zfs.packages
19:57 <Shiz> and handle those through apk
19:57 <Shiz> in that case
19:57 <TemptorSent> kaniini: Supporting existing configurations so I don't get drawn and quarted when everything breaks with a new update-kernel script!
19:57 <kaniini> update-kernel is an alpine tool
19:57 <TemptorSent> Shiz: All of that has already been reimplemented.
19:57 <kaniini> if it breaks for somebody who is doing things in an unexpected way
19:57 <kaniini> then that's on them
19:57 <TemptorSent> kaniini: Yeah, have you looked at it yet?
19:57 <kaniini> why do i need to look at it
19:57 <Shiz> TemptorSent: btw not trying to disparage your work or anything
19:58 <Shiz> just trying to understand it
19:58 <TemptorSent> kaniini: Because the mess is the interaction between update-kernel and mkinitfs.
19:58 <kaniini> i still do not understand why you would use update-kernel or mkinitfs with a custom build that isn't also managed by apk
19:58 <TemptorSent> Shiz: Take a look in the root of my branch -- there are a couple of README files.
19:58 <Shiz> url?
19:59 <TemptorSent> kaniini: Because you may only need to custom-build one set of modules for your custom-hardware!
19:59 <TemptorSent> Shiz: https://github.com/TemptorSent/aports/tree/mkimage-refactor-scripts/scripts/mkimage
20:00 <TemptorSent> kaniini: Or you may be building the stock -grsec kernel with a few added patches and everything else from the main dist, including the modules.
20:01 <kaniini> see your second example is stupid
20:01 <kaniini> because if you're doing that
20:01 <kaniini> you would probably just make a copy of it out of aports
20:01 <kaniini> and build it the alpine way
20:01 <Shiz> the second example is also flawed
20:01 <Shiz> you should not use kernels and modules build using different source trees or configs
20:01 <Shiz> it is not guaranteed to work
20:02 <TemptorSent> Shiz: Of course not, but I can think of plenty of perfectly valid use cases, including building in your own keys for runnign on a secboot config.
20:02 <Shiz> which is a different config
20:02 <Shiz> and secure boot is not handled in the linux kernel, but in the bios
20:03 <TemptorSent> Shiz: The kernel must have the keys to validate the modules.
20:03 <kaniini> Shiz: not exactly
20:03 <Shiz> point being, imo if you're building your own kernel, do not rely on distro kernel packages
20:03 <Shiz> TemptorSent: that's not secue boot, just module signing
20:03 <TemptorSent> Shiz: So if you want to use custom modules, you have to build in the keys for them.
20:03 <kaniini> Shiz: if secure boot is requested, you are supposed to go all the way down
20:03 <kaniini> Shiz: module signing, etc
20:03 <Shiz> kaniini: 'supposed to'
20:03 <Shiz> p
20:03 <Shiz> :p
20:03 <TemptorSent> Shiz: it's required by the spec.
20:04 <Shiz> anyway, again
20:04 <kaniini> Shiz: if we would like our bootloader signed, we have to agree to follow the spec
20:04 <Shiz> using a custom kernel but distro kernel module packages
20:04 <Shiz> is a bad idea
20:04 <Shiz> imo
20:04 <kaniini> yes
20:04 <TemptorSent> Shiz: Point is, adding keys to a kernel would require building it.
20:04 <Shiz> kaniini: good thing we don't, as far as I know
20:04 <kaniini> this is what i have been saying for the past hour
20:04 <Shiz> kaniini: well i don't consider mkinitfs a distro kernel package
20:04 <Shiz> just the modules files
20:04 <Shiz> so there may be some difference there
20:04 <Shiz> :p
20:05 <kaniini> Shiz: EFI secure boot support is something we actually do desire
20:05 <TemptorSent> I'm just trying to put the required information in place so we can act on it when we get there.
20:06 <Shiz> as iPXE has stated: http://ipxe.org/appnote/etoken#waiting
20:06 <Shiz> :p
20:06 <TemptorSent> Like knowing which apk we need to check the signature on to verify that we in fact have the right file in our boot.
20:07 <Shiz> i also don't really see how you aim to guarantee mkinitfs integrity from within mkinitfs
20:07 <Shiz> err
20:07 <Shiz> s/mkinitfs/initramfs/g
20:07 <TemptorSent> Also, we don't need M$'s tokens for secure boot, we can install our own.
20:07 <Shiz> surely if i can compromise the initramfs i can patch out your /init or apk code that checks sigs
20:07 <TemptorSent> Shiz: The stub would live in the kernel, baked in
20:08 <Shiz> okay, same thing
20:08 <TemptorSent> Shiz: Which includes apk.static and busybox.static.
20:08 <Shiz> if i can compromise your initramfs
20:08 <Shiz> i can also very likely compromise your kernel
20:08 <Shiz> since they're 99.9% of the time stored together
20:08 <kaniini> TemptorSent: it is desirable to have release media signed in such a way that it "just works" (tm)
20:08 <TemptorSent> Shiz: It's the payload of the initramfs that's the issue.
20:09 <TemptorSent> kaniini: Agreed, but it's not required for the featuer to be useful.
20:09 <Shiz> i don't fully understand your threat model i guess
20:09 <Shiz> my idea is that everything that would lead to the need to check initramfs integrity also trivially leads to kernel compromise
20:09 <Shiz> which means checking the integrity is useless from within the initramfs
20:09 <Shiz> baked-in or not
20:09 <TemptorSent> Shiz: You can add anythign you want by appending it to the end of a cpio and the kernel will keep reading it.
20:10 <Shiz> yes, and?
20:10 <Shiz> if i can append, i can modify
20:10 <Shiz> if i can modify, i can pwn your kernel
20:10 <Shiz> :p
20:10 <Shiz> initramfs integrity is useless if i can just run my botnet in your ring0
20:10 <TemptorSent> Shiz: So the attack just has to keep spitting out data without having to alter anythign in line.
20:10 <kaniini> and if the bootloader just checks the checksum of the initramfs (i.e. signatures), then none of this is needed
20:10 <Shiz> TemptorSent: this distinction is not meaningful in any real world scenario that i know of, though
20:11 <kaniini> the way it should be is
20:11 <TemptorSent> Shiz: netcat :)
20:11 <kaniini> bios checks bootloader, bootloader checks kernel and initramfs
20:11 <Shiz> what about it?
20:11 <Shiz> netcat is just tcp
20:11 <kaniini> initramfs checks real /sbin/init
20:11 <Shiz> if i can add data to your tcp connection, i've intercepted it in the first place
20:11 <Shiz> meaning i can also just rewrite data
20:11 <TemptorSent> Shiz: Say you're tftp booting, and someone spits a bunch of crap out to the end of the stream.
20:12 <Shiz> it doesn't work like that, though
20:12 <kaniini> unless you mean
20:12 <TemptorSent> Shiz: tftp is very easy to spoof.
20:12 <Shiz> yes
20:12 <Shiz> it is very easy to spoof
20:12 <Shiz> very easy to just modify packets too
20:12 <Shiz> you don't need appending for it
20:12 <TemptorSent> Shiz: So everything looks fine.
20:12 <Shiz> again, the difference between appending and modifying is not meaningful here
20:13 <TemptorSent> Shiz: The files all look right, etc on inspectino
20:13 <Shiz> they are all trivially doable
20:13 <Shiz> and if i can modify the files, i can compromise your kernel
20:13 <Shiz> rendering the entire integrity check useless
20:13 <kaniini> which is why the bootloader has to be responsible for verifying it
20:13 <Shiz> yes
20:13 <kaniini> not the initramfs itself
20:14 <TemptorSent> Shiz: The kernel has signature ability.
20:14 <kaniini> that's not good enough
20:14 <Shiz> yes, which i'll gladly patch out if i can modify the kernel
20:14 <kaniini> because we can boot a kernel with that patched out
20:14 <Shiz> now what?
20:15 <TemptorSent> Forget it. Apparently there is no point in making life hard for script kiddie.s
20:15 <Shiz> i think a script kiddie would rather just replace the entire kernel immediately
20:15 <Shiz> instead of doing weird append tricks
20:16 <Shiz> security has a cost, and i don't mean just technical cost by that
20:16 <Shiz> it has a social cost
20:16 <TemptorSent> Shiz: Replacing the entire kernel tends not to work if you do sane things with your bootloader to at least look at a checksum
20:16 <Shiz> not true
20:16 <Shiz> checksums are trivially brutable
20:17 <kaniini> if your bootloader is checking signatures
20:17 <kaniini> then it is fine
20:17 <Shiz> and if you look at a checksum, appending's not gonna work either
20:17 <Shiz> (continuing from where I left...)
20:17 <Shiz> security has a social cost because if we add an integrity check like this which is not actually secure
20:17 <Shiz> but people THINK they are secure
20:17 <Shiz> they may forego more effective security measures
20:17 <kaniini> it means those people eventually die
20:17 <Shiz> the only thing worse than no security is false security
20:17 <TemptorSent> Shiz: That's the problem with the way the kernel reads, you can have the checksumed data exactly the same, then append to that and the kernel reads it transparently.
20:18 <Shiz> TemptorSent: you just said your bootloader
20:18 <kaniini> because the NSA got an implant on their machine
20:18 <kaniini> and the cruise missile is on the way
20:18 <Shiz> which is what iw as talking about
20:18 <Shiz> also
20:18 <TemptorSent> Shiz: iPXE for instance.
20:18 <Shiz> believe me, i know plenty about ipxe
20:18 <Shiz> if you checksum your kernel, you also checksum your initramfs
20:18 <kaniini> ipxe can check signatures
20:18 <Shiz> in which case appending/modification wouldn't work either
20:18 <Shiz> and yes, ipxe can check asymmetric signatures
20:19 <Shiz> i have that working in production actually
20:19 <Shiz> :p
20:19 <TemptorSent> Shiz: I believe that you can still get the kernel to swallow additional append data.
20:19 <kaniini> as in, what?
20:19 <Shiz> you can not
20:19 <kaniini> kernel foo
20:19 <kaniini> append bar
20:19 <kaniini> append evilbar
20:19 <kaniini> ?
20:19 <TemptorSent> Shiz: It's a broken part of the kernel's loading that it keeps processing cpio data until the data stream ends.
20:19 <Shiz> nothing that the bootloader can't check anyway
20:20 <Shiz> TemptorSent: just like how the bootloader checks checksums
20:20 <Shiz> so?
20:20 <Shiz> i'd be very afraid if my bootloader did cpio parsing
20:20 <Shiz> :p
20:20 <TemptorSent> Shiz: All the bootloader does is appends them all together and hands it to the kernel.
20:21 tmh1999 joined
20:21 <Shiz> i think you're getting lost in your own argument
20:21 <Shiz> you claim you can prevent kernel modifications by checking the checksum
20:21 <Shiz> to which i say 'you can also check the initramfs checksum'
20:21 <Shiz> (in the bootloader)
20:21 <Shiz> how does that statement oppose that?
20:21 <TemptorSent> Shiz the CONTENTS of the initramfs are my concern, since the checksum says nothign about the origin of those files
20:22 <TemptorSent> Shiz: So I can't go back to something signed and verify them.
20:22 <Shiz> indeed, a checksum is rather useless
20:22 <TemptorSent> Shiz: Which I CAN do for the kernel
20:22 <Shiz> how?
20:22 <Shiz> you can just as simply make asymmetric signatures over the initramfs
20:22 <Shiz> that's what i'm doing right now in production
20:22 <TemptorSent> Shiz: Because that is actually installed by apk and tracked IIRC.
20:22 <Shiz> but you don't get it
20:23 <TemptorSent> Shiz: Where do the files in your initramfs COME from?
20:23 <Shiz> me, which is verified by the asymmetric signature over htem
20:23 <kaniini> TemptorSent: once update-kernel signs the initramfs image, it is surely good enough no?
20:23 <TemptorSent> Shiz: That is currently the problem, it is built from files of unknown origin with no signaures or checksums.
20:24 <kaniini> no it's not
20:24 <Shiz> yes, but you don't get my point
20:24 <kaniini> it's built from files on your machine
20:24 <Shiz> the kernel is not the tiniest bit safer than your initramfs
20:24 <Shiz> i can modify your kernel file and you won't notice
20:24 <Shiz> sure, # apk upgrade will replace it, so what?
20:24 <Shiz> apk doesn't checksum and verify your system every time it's ran
20:24 <TemptorSent> Shiz: Will it match the sums when I go to install it?
20:25 <Shiz> install what?
20:25 <Shiz> it's already on your disk
20:25 <Shiz> and if you mean i can compromise the channel between you and your repos, yes
20:25 <Shiz> but that won't help me compromise the mkinitramfs either
20:25 <Shiz> since it's built locally on your machine
20:26 <TemptorSent> Shiz: I'm trying to ensure all files in the initramfs actually came from known sources so I can verify what's on the tftp server isn't tampered with!
20:26 <Shiz> but you can't
20:26 <Shiz> because the thing that verifies it is just as easily compromised as the thing it's verifying
20:26 <kaniini> actually,
20:26 <Shiz> it's fundamentally not real security
20:26 <kaniini> the channel between you and your repos
20:26 <kaniini> would fail to be compromised
20:26 <kaniini> as checksum mismatch :P
20:26 <TemptorSent> Shiz: Why not? I have a manifest of all included files, the apks they came with, and the versions.
20:27 <Shiz> TemptorSent: "what is checking it"?
20:27 <Shiz> the kernel booting?
20:27 <Shiz> too bad, i can compromise that just as easily
20:27 <Shiz> same goes for the initramfs stub in the kernel
20:28 <TemptorSent> Does nobody do what I used to do 25 bloody years ago and actually store a manifest of checksums offline to verify against?
20:28 <Shiz> if you do that, why not do the same for the initramfs in the first place?
20:29 <TemptorSent> How the hell do you verify your system if you don't have any idea of what you put there.
20:29 <Shiz> you use signatures
20:29 <Shiz> and have your bootloader check whatever it's loading
20:29 <Shiz> (like iPXE)
20:29 <TemptorSent> Shiz: That's what I'm trying to accomplish - a set of manifests that can relate a file back to its original -SIGNED- source.
20:30 <Shiz> so...
20:30 <TemptorSent> While still allowing people to only include the features they actually need in their initfs.
20:30 <Shiz> you're re-implementing # openssl cms?
20:31 <Shiz> i'm still not sure how this helps
20:31 <Shiz> why not sign the generated file instead
20:31 <TemptorSent> So if we have a stub that only loads signed files, and we load it with a signed kernel, we can be sure we have at least the same files we put there.
20:31 <Shiz> yes but
20:31 <Shiz> if you have a signed kernel that you can verify is signed by the bootloader
20:31 <Shiz> that very same bootloader can also verify the initramfs
20:32 <Shiz> which is my point...
20:32 <TemptorSent> Shiz: because then we have to either build, sign, and distribute every permutation of initramfs, or we have to have the user have a secure keychain and sign them himself every time.
20:32 <TemptorSent> Shiz: So how do you make an initramfs with say zfs modules included?
20:32 <TemptorSent> (and userspace files, which is actually the real issue, as modules can be signed)
20:34 <Shiz> mkinitfs ; openssl cms -sign -binary -noattr -signer file.crt -inkey file.key -in initramfs -out initramfs.sig
20:34 <TemptorSent> I want to be able to append a cpio with the the "initfs-fs-zfs-0.6.5.9-r0^4.9.19-0-grsec.apk" and have zfs support
20:34 <Shiz> remove the sig, append, re-sign it
20:34 <TemptorSent> Shiz: I was already shot for expecting users to create their own keys.
20:35 <kaniini> just use checksums to verify the initramfs
20:35 <kaniini> it is allowed by secure boot spec
20:35 <TemptorSent> kaniini: Where do you get the list of checksums?
20:35 <kaniini> inside the config file for the bootloader
20:36 <TemptorSent> (...and this is coming right back to exactly what I'm trying to implement, namely manifests with checksums)
20:36 <TemptorSent> kaniini: That's where you PUT the list of checksums, I'm trying to solve where you GET them and how you make sure those files match the ones that came out of your original package.
20:36 <kaniini> sign it using the host key inside the system TPM
20:36 <kaniini> i dont know what to tell you
20:37 <Shiz> you get them by generating them?
20:37 <kaniini> i know that this is a solved problem, it may be worthwhile to look at fedora or similar
20:37 <TemptorSent> Shiz: The apks already contain SHA1 checksums in the pax headers of the apks.
20:37 <Shiz> and?
20:37 <Shiz> who cares
20:38 <Shiz> you get the checksum by generating it when building the initramfs
20:38 <Shiz> of the entire initramfs file
20:38 <TemptorSent> I just don't understand why I'm getting killed for wanting to generate manifests of files installed in ways APK doesnt' track.
20:38 <TemptorSent> Shiz: Please look at how mkinitfs works.
20:38 <Shiz> because it seems useless
20:38 <kaniini> https://bentley.link/secureboot/
20:38 <Shiz> we aren't discussing how mkinitfs works right now though
20:38 <Shiz> we're discussing how we want it to work in the future
20:38 <TemptorSent> Shiz: mkinitfs knows nothing about the source of its files.
20:38 <Shiz> and that's fine
20:38 <Shiz> it's literally not relevant
20:39 <TemptorSent> Shiz: You are far more trusting than I then.
20:39 <Shiz> no, not really
20:39 <Shiz> i just know how to threat model
20:39 <Shiz> :p
20:39 <Shiz> if i can put untrusted files on your rootfs
20:39 <Shiz> i can add my keys to your /etc/apk/keys folder
20:39 <Shiz> and have you install my pwned <any package>
20:39 <TemptorSent> Shiz: I want to know that the files I install haven't been modified since they were unpacked from signed distfiles
20:39 <Shiz> you will always have to trust the node that generates the mkinitfs
20:40 <Shiz> TemptorSent: then run # apk verify before you generate
20:41 <TemptorSent> Shiz: That's what I'm doing.
20:41 <Shiz> i mean, i'm unclear WHERE do you want to check this
20:41 <Shiz> on the node that booted the initramfs? if that got fucked with ,that node's already compromised
20:41 <TemptorSent> Shiz: I don't want it to be able to pull a file from the filesystem that may be modified without the user being aware of it.
20:41 <Shiz> on the nod ethat generates the initramfs? # apk verify
20:42 <Shiz> then i don't see why you need anything but # apk verify
20:42 <TemptorSent> Shiz: The contents of mkinitfs don't come out of apks currently
20:42 <Shiz> doesn't matter
20:42 <Shiz> they come from the root filesystem
20:42 <Shiz> which is verified with # apk verify
20:42 <TemptorSent> Shiz: It doesn't work quite that way.
20:43 <TemptorSent> Shiz: The problem is it's fed a glob pattern, which you can't exactly apkverify
20:43 <TemptorSent> Shiz: Which is the problem I'm solving by creating the manifests.
20:44 <TemptorSent> You glob against the manifest, not the filesystem.
20:44 <Shiz> but.. you can?
20:44 <Shiz> # apk verify $(find / -name $glob)
20:44 <TemptorSent> Shiz: Try it, let me know what you get :)
20:45 <TemptorSent> And we're talking more than one or two globs in many cases.
20:46 <Shiz> something that tells me there is likely a bug in # apk verify that should be fixed?
20:46 <TemptorSent> And still, that leaves you with files you can't track after the fact.
20:46 <TemptorSent> Actually, I think you want audit :)
20:46 <TemptorSent> verify only checks the .apk
20:46 <TemptorSent> But the problem is the way audit stores it's information.
20:47 <TemptorSent> It can't tell you if an arbitrary file matches it's original, only if a given file at a given install path has been modified.
20:47 <Shiz> but that's the same question, no?
20:47 <TemptorSent> So you can't verify individual files out of context.
20:47 <Shiz> apk audit | cut -d' ' -f2- | grep myfilepath
20:48 <Shiz> (this is where you can apply the glob too)
20:48 <Shiz> or without the cut: apk audit -q | grep myfilepath
20:48 <Shiz> since the files are being taken from the rootfs in the first place
20:49 <TemptorSent> Shiz: Okay, now how do we check that after the fact?
20:49 <Shiz> from where?
20:50 <TemptorSent> Shiz: So once some file has been copied elsewhere, we still know where it came from?
20:50 <Shiz> no, but i doubt the necessity of that
20:50 <TemptorSent> Shiz: Then how do you go back and do a verification?
20:50 <Shiz> you answer my question first
20:51 <TemptorSent> Shiz: You basically have no further information at that pont to work with.
20:52 <TemptorSent> Shiz: The simplest case is just simply detecting corrupt transfers.
20:52 <Shiz> that wasn't my question
20:52 <Shiz> my question was 'from where'
20:52 <TemptorSent> Verified from where or copied from where?
20:52 <Shiz> verified
20:53 <TemptorSent> From either the running system or from an external audit.
20:54 <Shiz> verifying from the running system is rather useless, as it has already been compromised if the initramfs is borked
20:54 <TemptorSent> From within the running system it at least gives you reasonable ability to determine if you're running your expected config.
20:54 <Shiz> verifying from external audit, why does it matter where the individual files came from if you can simply check the entire initramfs sig?
20:54 <TemptorSent> Shiz: Because knowing WHAT changed is important.
20:56 <TemptorSent> Shiz: And again, if we want to allow users to load only what they want in the initramfs without having them have to create and secure keys, we need a means of adding arbitrary content to the initramfs, but only loadign that which matches our keys.
20:57 <TemptorSent> Shiz: So if we need to extract say /usr/sbin/zpool and /usr/sbin/zfs from the zfs package to install in the initramfs, it would be much more useful to have a signed manifest to include than requiring the entire zfs apk to be included!
20:58 <TemptorSent> Or, at the very least, a reference to the signed apk that can be used to verify the checksums after the fact if needed.
21:00 <TemptorSent> Shiz: Ideally, the next APK version would support signed manifests directly :)
21:00 <TemptorSent> The pax headers are a cool idea, but are very hard to use for anything but verifying the apk itself.
21:01 <TemptorSent> The information that audit runs against is not signed in any way IIRC.
21:01 <TemptorSent> So we end up having to trust something that we can't verify without downloading every single apk on the system.
21:01 <TemptorSent> ...and that's the issue I'm trying to fix.
21:02 <TemptorSent> Or at least reduce in scope to checking a manifest to find which apk we need in order to verify.
21:03 <TemptorSent> And being able to distinguish two versions of the same file by their checksum.
21:05 <TemptorSent> For verification purposes, every file on the system should be able to be traced back to its point of origin and those signatures checked against a RO keychain.
21:07 <TemptorSent> It's not just a matter of runtime security, but largely one of enabling forensics and mitigation.
21:09 <TemptorSent> This is ESPECIALLY important if an upstream package is ever comprimised, as it gives us a fingerprint we can use to send revocation certificates against (ideally, packages would get signed with their own key which is in turn signed by the alpine key so we could blacklist individual builds if they get comprimised.
21:14 <TemptorSent> Shiz: I've spent a lot of time doing forensic analysis after systems have been attacked, sometimes successfully, and sometimes not. The hardest thing to do is figure out what, if anything, was altered, and HOW.
21:27 <TemptorSent> Okay, concrete example using iPXE: The kernel is a stock kernel, verified against the alpine keys from a WKS, the mkinitfs image itself is user-generated, requiring either that the user sign the image and upload the signature to a keyserver, or that untrusted initrd images be allowed by iPXE and the init stub in the verified kernel verifys the contents of anything appended.
21:30 <TemptorSent> (in ipxe script) kernel http://untrusted.com/vmlinuz && imgverify vmlinuz https://trusted.com/keys/vmlinuz.sig && boot vmlinuz
21:30 <TemptorSent> With that, and the rest as-is, we can have at least a mostly-trusted boot if you trust your pxe flash.
21:32 <TemptorSent> Can we agree that given that, and a stub which verifies the contents of the initramfs, we have a boot that's at least as trustworthy as our pxe rom and the signatures?
21:34 <TemptorSent> And that that would eliminate a lot of low-hanging fruit for exploits which can alter data in-flight, but non-transparently?
21:35 <TemptorSent> As well as provide a much better framework for auditing and forensics than having files with no information tracked?
21:37 blueness joined
21:43 <Shiz> sorry, i was kinda busy with stuff
21:43 <Shiz> i'll read the scrollback in a bit
21:47 TemptorSent joined
22:15 leitao joined
22:20 gromero_ joined
22:50 blueness joined
23:24 ostera joined
23:32 ostera joined
23:34 blueness joined
23:52 rdutra joined
23:59 ostera joined