<    March 2017    >
Su Mo Tu We Th Fr Sa  
          1  2  3  4  
 5  6  7  8  9 10 11  
12 13 14 15 16 17 18  
19 20 21 22 23 24 25  
26 27 28 29 30 31
00:00 <TemptorSent> alright, I'm going to leave the base layout as it is for now and push the beginnings of my profile-surgery.
00:01 <tmh1999> Sorry I can't be helpful for now. I just started picking up on building the boot image. Lots of stuffs to read. But I marked your things above.
00:01 <TemptorSent> So, do we really need the keymaps in the base system?
00:02 <TemptorSent> tmh1999: Yeah, the boot image building is currently about half voodo... that needs to be fixed, but I don't have time to do that this week.
00:04 <TemptorSent> tmh1999: Right now, mkimage actually seems to do the right thing as far as its handling of the boot stuff.
00:04 <TemptorSent> tmh1999: What boot scheme does s390 want?
00:05 <tmh1999> TemptorSent : typically it just need a kernel image and and initrd image. That's what I know till now.
00:06 <TemptorSent> Okay, and how does it configure the boot?
00:07 <tmh1999> I am still figuring it out
00:08 <TemptorSent> Okay, let me know what it needs and we'll see if we can't add a generator in mkimage
00:19 <TemptorSent> tmh1999: Most recent revision pushed has profile surgery -- take a look, I think it might be straightforward to add the s390 target once you have a viable kernel.
00:27 <tmh1999> Thanks for the hint
00:46 <TemptorSent> It looks like someone tagged lddtree 1.26 *YAY* Now, let's see if it works.
00:50 <TemptorSent> Well, it doesn't break mkinitfs, so I think we're good. Thank you!
02:19 blueness joined
02:25 s33se_ joined
02:44 Emperor_Earth joined
03:05 jud_ joined
03:33 iamthemcmaster_ joined
03:35 systmkor_ joined
03:36 NightKhaos_ joined
03:39 chris| joined
03:43 Ganwell joined
03:46 blueness joined
03:49 al3hex joined
03:49 victorbjelkholm joined
03:51 <TemptorSent> Is anyone aware of a current means of doing a 'run-once' script on first startup currently, or shall I hack one together?
03:52 <TemptorSent> (The preceeding question was brought to you by the department of redundancy department)
05:25 <kaniini> there isnt really any support for firstrun
05:35 <TemptorSent> kaniini: Oh well, guess I get to make something.
05:35 <TemptorSent> Or at least a runonce that offs itself...
05:35 <TemptorSent> I'm thinking /var/local/runonce.d
05:38 <kaniini> TemptorSent: linux on s390 runs inside a VM usually fwiw
05:38 <kaniini> if youve ever used Xen it is a lot like that
05:39 <TemptorSent> kaniini: Okay, got it -- anything special needed there in terms of config file generation/driver loading/ etc?
05:39 <TemptorSent> kaniini: Yeah, xen is giving me fits... I'm likely to go back to KVM for most applications I think.
05:40 <TemptorSent> virtio+9p+kvm is looking pretty good right now.
05:41 <TemptorSent> I don't need the extras of xen for most of my workloads.
05:41 <TemptorSent> kaniini: If someone wants to give me some basics for a s390 profile, I'll add it to mkimage
05:42 <TemptorSent> Right at the moment, I'm working on getting postgres to auto-start and auto-load a dump.
05:46 <TemptorSent> I think I have everything in there now that I need to load a database from media on the first run.
05:49 <kaniini> TemptorSent: i am not sure
05:50 <kaniini> i used to have a s390 mainframe (appexpress or something like that), but i got rid of it when i moved several years ago
05:50 <kaniini> you have to use another system to actually control it
05:50 <kaniini> the machine that came with mine ran OS/2
05:51 <kaniini> so that should give you an idea of how weird this stuff is :P
05:54 <TemptorSent> *lol* Was it WarpDesktop?
05:55 <kaniini> idk
05:55 <kaniini> it had something that looked like windows 3.1 program manager
05:56 <kaniini> i dont think it was a full OS/2 install
06:06 <TemptorSent> Flashbacks.
06:38 <TemptorSent> kaniini : Anyway, it'd be nice to support it if we can without much pain
06:48 alacerda_ joined
07:48 <TemptorSent> Hmm, replacing config files after they're installed is a PITA when you have to edit the ones provided in the APKS...
07:56 czart__ joined
08:20 <TemptorSent> Comments on installing self-destructing run-once hooks into /var/local/hooks/?
08:21 <TemptorSent> The post-extraction/pre-rc hook will probably require minor changes to init, but the remainder should be doable through standard /etc/runlevel entries
08:23 <TemptorSent> It's fairly necessary for run-from-ram setups or automated installs.
08:38 <TemptorSent> Would hacking inittab be cleaner?
09:07 <skarnet> no
09:07 <skarnet> hacking inittab is never clean
09:08 <skarnet> whatever you do, please make sure it's as independent from the init system and service manager as possible
09:09 <skarnet> the init scripts should end with a oneshot that looks into a run-once hooks directory, runs stuff there, and deletes it if the subprocess exits 0
09:10 <TemptorSent> skarnet: It seems like it may be in this instance, we can do a mount --bind /our/tmp/inittab /etc/inittab before pivot, and have the last thing it does before starting openrc is unmounting the bind and hupping itself.
09:10 <skarnet> so you would just have to add your run-once hooks to that directory.
09:10 <skarnet> what part of "never clean" don't you understand
09:10 <TemptorSent> skarnet fair enough...
09:11 <skarnet> hupping init is a nest of vipers. I wanted to do it in my own apk, to get something supervised by init instead of launched by openrc
09:12 <TemptorSent> Hmm, it works for me...
09:12 <skarnet> for every problem it solved, it created 2
09:12 <TemptorSent> What sort of issues did you run into?
09:13 <skarnet> first, obvious thing is that it won't work anymore when we replace init :P
09:13 <TemptorSent> There is that :)
09:13 <skarnet> and the day may be closer than you think
09:14 <TemptorSent> One problem is that from the initrd init process, the sysroot doesn't get pivoted to until the exec call to init.
09:14 <TemptorSent> So prepending a "preinit" runlevel would be an easy, and relatively clean fix.
09:15 <skarnet> you want to run a script at the *end* of the init sequence, don't you?
09:15 <TemptorSent> If there's anything in that directory, it runs.
09:15 <skarnet> it sounds like you want to run stuff *before* init
09:15 <TemptorSent> I want after apks extracted/overlays applied, but before init on the sysroot.
09:15 <skarnet> please clarify exactly when you want to run your stuff
09:16 <kunkku> I usually use a normal init script for oneshot tasks
09:16 <kunkku> then store flag file e.g. in /var/lib to indicate completion
09:17 <TemptorSent> Essentially, I want to apply modifications to the root fs AFTER the initrd init completes and pivots root, but BEFORE the services start with /sbin/init.
09:17 <kunkku> with this approach, you can leverage the init systems dependency management
09:18 <TemptorSent> I need a "run-me-first" essentially for it to work that way.
09:18 <skarnet> TemptorSent: what prevents you from adding your stuff at the end of the initramfs's /init before the exec /sbin/init ?
09:18 <TemptorSent> skarnet: The pivot-root takes place as the exec call to /sbin/init.
09:19 <skarnet> wat
09:19 <TemptorSent> skarnet: take a look at /usr/share/mkinitfs/initramfs-init
09:21 <skarnet> yeah, so what
09:21 <TemptorSent> skarnet it does an "exec /bin/busybox switch_root $sysroot $chart_init /sbin/init $KOPT_init_args"
09:21 <skarnet> if you want to perform something between switch_root and /sbin/init, insert a script there
09:22 <TemptorSent> True, not impossible... but it only kills one bird with the stone, and might break things worse.
09:23 <TemptorSent> I'll have to see how the logistics for that would work out.
09:24 <skarnet> the question is, do you have a rw filesystem after you switch_root
09:24 <skarnet> where you can actually write your runonce script
09:24 <skarnet> with a tmpfs rootfs, it's trivial
09:24 <TemptorSent> skarnet: That's why the bind mount (or tmpfs)
09:25 <skarnet> yeah, you can create a tmpfs in $sysroot/runonce, write your script there, and unmount it afterwards
09:25 <TemptorSent> What I'm tryign to do is modify config files and add users between the extraction of the rootfs and the execution of the services.
09:26 <skarnet> oh, then your rootfs is rw no matter what
09:26 <TemptorSent> skarnet: Right -- what I was hoping to do was mount it to /etc/runleves/preinit and let inittab handle the magic.
09:26 <skarnet> if you're gonna modify the rootfs on the fly anyway (which is bad but I guess it's ok for the first run) then there's no problem
09:27 <TemptorSent> skarnet: Think run-from-ram systems.
09:27 <skarnet> I'm precisely thinking *other* systems
09:27 <skarnet> run-from-ram systems are easy
09:27 <TemptorSent> skarnet: They'll ALWAYS have a fresh image needing modification.
09:27 <skarnet> "let inittab handle the magic" = delegating the work to the init system, which isn't nice to me
09:27 <skarnet> the clean solution is
09:28 <TemptorSent> The problem is when you have a system that mounts root from an external source.
09:28 <TemptorSent> ...you only want to do your config once, and once you've used lbu or whatnot, you really don't want to be overwriting things again.
09:29 <skarnet> mount -t tmpfs tmpfs $sysroot/runonce; cp mypreinit /runonce/preinit ; exec /bin/busybox switch_root $sysroot %chart_init /runonce/preinit
09:29 <TemptorSent> more to the point, let openrc do it's thing just like it normally would.
09:29 <skarnet> and /runonce/preinit does its stuff and unmounts the tmpfs
09:30 <skarnet> I meant cp mypreinit $sysroot/runonce/preinit obviously
09:30 <skarnet> and preinit execs into /sbin/init at the end
09:30 <TemptorSent> Right, but how do I inject the preinit in the initrd init cleanly with that approach?
09:30 <skarnet> you want to insert stuff between initramfs and init, that's where you do it
09:31 <skarnet> it's a script, you can have it as a heredoc document in your initramfs /init, or anywhere else in the initramfs
09:31 <TemptorSent> What I really want a pre-sysinit runlevel I guess.
09:31 <skarnet> that's exactly what I'm giving you
09:31 <skarnet> a script running as process 1 in the real rootfs
09:31 <skarnet> that execs into init at the end
09:32 <skarnet> stop thinking in terms of runlevels
09:32 <TemptorSent> skarnet: Understood, like I said, that kills the preinit bird..
09:32 <skarnet> think in terms of what's really happening
09:33 <TemptorSent> skarnet: Then I need to run another set of scripts once the system is started (think database loading)
09:33 <skarnet> that's a normal service you can let the service manager run at the end
09:33 <TemptorSent> Hence the reason I was eyeballing inittab.
09:33 <skarnet> consider inittab a hot poker, unsuitable for your eyeballs
09:34 <TemptorSent> Right - does OpenRC give you a first/last dep?
09:34 <skarnet> there should be a rc.local service or something like that
09:34 <skarnet> if there isn't, we should add one
09:35 <skarnet> run your "early" stuff before executing /sbin/init, run your "late" stuff as a normal part of the service manager
09:35 <skarnet> so you'll be entirely independent from the init system *and* the service manager
09:37 <TemptorSent> If there was a pre-sysinit runlevel, that would take care of it, and possibly be useful elsewhere.
09:37 <skarnet> PLEASE stop thinking in terms of runlevels
09:38 <skarnet> runlevels are semantically vague
09:38 <TemptorSent> Okay, service manager dep then.
09:38 <skarnet> "before sysinit" means "the first thing the system should run at boot time"
09:38 <TemptorSent> In terms of how busybox's init works, it's pretty simple.
09:39 <TemptorSent> before sysinit means the first thing the system should do after initrd has given over control
09:39 <skarnet> yes
09:39 <skarnet> and that's exactly what you do in a pid 1 script that execs into your real init when it's done
09:39 <TemptorSent> Before starting system services, but from within openrc would be ideal.
09:42 <TemptorSent> "::sysinit:/sbin/openrc initfirst" would do it.
09:43 <skarnet> You're designing something at a *lower level* than the service manager and even init, and you're still thinking of terms of init and openrc
09:43 <TemptorSent> skarnet: Part of the problem is this needs to be fairly general.
09:43 <TemptorSent> ?? What do you mean at a lower level?
09:44 <TemptorSent> The overlays work by adding the links for the runlevels.
09:44 <skarnet> you're doing stuff in the initramfs and system installation.
09:44 <skarnet> This is more bare-metal, has more privileges, than even init.
09:45 <skarnet> trying to contort your stuff in order to fit in the init/openrc boxes is reversing the abstraction.
09:45 <TemptorSent> skarnet: Think apkovl.
09:45 <TemptorSent> skarnet: It configures a completely normal alpine system.
09:46 <TemptorSent> The only problem with an overlay is you need the files you want to operate on to exist first.
09:46 <skarnet> I'm trying to talk software design and approach to system stuff and you keep dragging me back to specifics.
09:46 <TemptorSent> Say I want to add a set of users based on a file retrieved from the web.
09:47 <TemptorSent> Can I even expect things like su and adduser to work before I'm in my real init under busybox?
09:48 <skarnet> If you're in your real rootfs, yes you can
09:48 <TemptorSent> What happens with the exec/pivot root then? Are my open files orphaned or ??
09:49 <TemptorSent> That's where it gets sticky from where I'm looking at it.
09:49 <skarnet> um, don't have open files when you switch_root
09:49 <skarnet> that's a given
09:49 <skarnet> pivot_root wouldn't be a problem, but switch_root is more complicated
09:50 <TemptorSent> skarnet: Then I'm going to have to figure out how to setup a context that I can get away with it.
09:50 <skarnet> also, I don't want to get into specifics, but "Say I want to add a set of users based on a file retrieved from the web" elicits a strong "why on earth would I want to do that" reaction.
09:50 <TemptorSent> As it stands, proc refuses to unmount from the sysroot.
09:51 <TemptorSent> skarnet: Grabbing configs for an automatically provisioned system.
09:51 <TemptorSent> skarnet: They all have unique keys, but the inidivual configurations may not be entirely determined at deployment time.
09:51 <skarnet> yeah, no, you don't do that before init.
09:52 <skarnet> you boot a minimal system and THEN grab a config.
09:52 <TemptorSent> skarnet: So they grab a signed package, add the users, modify settings, etc.
09:52 <TemptorSent> Right, the problem is I may need the users existing before booting the system in some cases.
09:53 <skarnet> those cases are broken, sorry.
09:54 <skarnet> a minimal system really doesn't need that many users.
09:54 <TemptorSent> skarnet: They need to work for SSH in my cases.
09:55 <TemptorSent> skarnet: In some cases, it's even required for them to establish ANY network connection.
09:56 <skarnet> you're talking about users and ssh and config - all that's fine, but it has nothing to do with initramfs and running stuff before sysinit.
09:56 <TemptorSent> skarnet: So unless I want to hack the ssh server as well, my best option is to add the users home directory/keys/etc to the overlay, then add the user to the system at the earliest opportunity.
09:56 <skarnet> You have several classes of problems and they need to be addressed separately.
09:57 <TemptorSent> skarnet: Um, if I need it to actually mount what I need for services, it becomes a catch-22.
09:59 <TemptorSent> skarnet: What I have right now is a client that I'm two weeks late on getting automated images to, and the main thing holding me up is the inability to alter the config before the services I need to run.
10:00 <skarnet> ... you mean you're doing this because you have a job to do for a client ?
10:00 <TemptorSent> skarnet: Modifying inittab and placing the modified copy in the overlay will take me a few minutes, and doesn't require modifications to any other packages to work.
10:00 <TemptorSent> skarnet: Among other things, yes.
10:01 <skarnet> how about you make something work for you and your client, and you don't try to rush it into Alpine mainline before we get the time to fully think it over and properly design it?
10:01 <TemptorSent> skarnet: I have unix-inept users that I'm distributing database servers to that need fully automated configuration.
10:02 <TemptorSent> skarnet: I intend to, but I'd rather put my efforts towards something that works for the general case, not just this particular project.
10:02 <skarnet> you can't do that when you're late.
10:02 <TemptorSent> I also need much the same functionality for a rpi based WX station project.
10:02 <skarnet> Douse your fire.
10:02 <skarnet> We'll think about the general case when you don't have external pressure.
10:03 <TemptorSent> skarnet: Yeah, most of the being late was my back going out and not being able to sit at a terminal for a couple weeks.
10:03 cyteen joined
10:04 <TemptorSent> skarnet: The project has been up and down on their priority list, so I don't feel entirely horrible and I'd rather have it done right than done fast and have to pick up the pieces later.
10:05 <TemptorSent> skarnet: I've already dedicated a fair bit of time to testing other configurations and making sure I wasn't breaking things, and hopefully have more working.
10:06 <skarnet> if you'd rather have it done right than done fast, why are you working on it on a Saturday night and why am I spending a good chunk of my Sunday morning discussing it when I'd rather be doing something else?
10:06 <TemptorSent> I'm tempted to just ship a static passwd file for now, since I have that working already.
10:06 <TemptorSent> Because this is when I can get work done :)
10:07 <skarnet> Ship your static passwd file, it's a KISS solution.
10:07 <TemptorSent> And I still need soon, just not rushed.
10:08 <skarnet> working on the week-ends screams rushed.
10:08 <TemptorSent> skarnet: Yeah, the database can get a version of the current config file to modify for the moment.
10:08 <TemptorSent> skarnet: I don't have a regular work week at all, so I tend to sit down and work when I feel like it.
10:09 <TemptorSent> skarnet: Insomnia lends itself to late-night coding.
10:09 <skarnet> so do I, but you need to understand most Alpine people have a schedule, and you can't take them hostage for live design discussions and reviews.
10:10 <skarnet> You also can't force them to keep pace with your schedule - "soon" may be too soon.
10:10 <TemptorSent> Understood.
10:11 <TemptorSent> I don't need anything done on that end really, I'm fine working out of my own tree -- I just want to make sure I'm actually making something useful.
10:11 <skarnet> that's why I'm advising you to work on a solution for your client, and come back to discuss integration when you don't have any external pressure and it's easier for other people to sync with you.
10:12 <TemptorSent> skarnet: The client's solution is essentially done as of this evening, excepting the configs.
10:14 <TemptorSent> skarnet: mkimage can build me one-off images with unique ssh keys, install all the apks I need, build the overlay, and get it all packaged as a bootable iso.
10:15 <TemptorSent> That solves my immediate need, but isn't quite fulfilling my long-term support goals of dumping a new image and walking away.
10:15 <TemptorSent> Oh, and it does zfs nicely, which is where the data will live, while the system itself runs from ram.
10:17 <TemptorSent> What I really want to work on once I get the client settled is getting the size of the image trimmed down by removing all but the required set of modules, esepcially for VM usage.
10:18 <TemptorSent> The virt image is currently 50MB, and 25% is modules.
10:18 <skarnet> don't mind my objections: my approach is one of minimalism (so, no zfs), having the rootfs on disk (not in ram), and I don't like the idea of initramfs in the first place (I'd do away with it entirely if I could), so we come from different places, and making Alpine work for all those cases is a challenge. ;)
10:19 <TemptorSent> skarnet: I'm using the run-from-ram setup for this application because it can't be broken beyond simply replacing the disc.
10:20 <skarnet> my pov is that a minimal read-only rootfs can't be broken either (and doesn't need an initramfs). But obviously not everyone agrees.
10:21 <TemptorSent> skarnet: I don't use initrds on most of my own installations because I have complete controll over them and can build a tailor-fit kernel with every driver I need and none I don't. In some cases, I run no modules at all!
10:21 <skarnet> yup, that's what I do too.
10:21 <skarnet> For installations you don't entirely control, it's harder.
10:22 <TemptorSent> skarnet: In ths case, the machine may not even have a SATA/SCSI hdd that I use, it may be ALL usb attached.
10:23 <TemptorSent> ...and that's where ZFS saves my ass -- they can stick a pair of usb2+ drives in, stick a CD in, and let it go.
10:23 <TemptorSent> ...I can zfs send the image to backup, and when they kill something, I just rebuild it from scratch and restore.
10:24 <TemptorSent> Even if the backup happens to be some windoze share.
10:26 <TemptorSent> skarnet: Yeah, if I wasn't a thousand miles away and dealing with people with more experience digging in dirt than in databases, it would be much easier.
10:27 <TemptorSent> As it is, I give them an iso to burn, they stick it in the drive, boot the system (with me talking them through the bios if necessary), and walk away.
10:27 <TemptorSent> Then they just drop reports in an incomming directory via sftp and get the result files back the same way.
10:29 <TemptorSent> All behind their firewall without needing any outside connections hopefully - that's one of the reasons I'm trying to get it right the first time.
10:29 <TemptorSent> I'll have provisions for establishing a remote-tunnel if needed, as a backup plan.
10:30 <TemptorSent> But yes, real life applications, coming soon.
10:32 <TemptorSent> Playing with code is fun and all, but I tend to go for the practical applications to make my life easier first.
10:35 <TemptorSent> You know what may solve the problem in a cleaner and more flexible way long term? Adding the ability for APK to run hook scripts stored in the apkovl before/after applying them.
10:37 <TemptorSent> In fact, that would solve MANY birds with one stone by keeping the apkovls split rather than one big combined overlay.
10:38 <skarnet> you'd need to talk to fabled and/or kaniini about this. I don't know much about apkovls. My first reaction is a red blinking light and a buzzer honking "warning: security risk" though.
10:39 <TemptorSent> Just put the initial stuff in the overlay loaded by the initrd, then have a service apply any overlays you want to do later.
10:39 <TemptorSent> skarnet: How so?
10:39 <TemptorSent> apks already have hooks that run on installation.
10:40 <skarnet> apks are signed.
10:40 <skarnet> Are apkovls signed too?
10:40 <TemptorSent> No reason you couldn't refuse to run scripts if they're not signed.
10:41 <TemptorSent> Or at least matching a pre-stored hash.
10:42 <TemptorSent> I'm not sure if the encrypted overlays are also signed or not, haven't looked at that yet.
10:45 <TemptorSent> The other intersting thing is that overlay mounts can have MULTIPLE lower members, so some of this may be simplifed by just adding each layer in sequence at the appropriate time.
10:45 <TemptorSent> (for run-from-ram, mostly)
10:48 <TemptorSent> This would let us easily build configuration deltas that could be backed up and replayed to a clean install.
10:49 <TemptorSent> (using lbu + whatever scripting we need for state information)
10:50 <TemptorSent> Anyway, I'm going to get some sleep with any luck, thank you for your time!
10:51 <skarnet> sweet dreams!
11:09 <clandmeter> Yes they are also signed
12:09 blueness joined
13:30 pickfire_ joined
14:11 LouisA joined
14:18 blueness joined
14:44 blueness joined
15:49 LouisA joined
18:24 nmeum joined
18:44 <kaniini> skarnet: apkovl are signed and optionally encrypted
18:55 blueness joined
19:21 <TemptorSent> kaniini: What are your thoughts regarding adding the ability to add pre/post scripts to apkovls?
19:23 <TemptorSent> And I'm still looking for a way to get abuild-sign to stop prompting me with mv -i for the index every time it gets regenerated.
19:33 BitL0G1c joined
21:35 <nmeum> ncopa: any ideas how can finally fix %2961? I encountered yet another package affected by this. I think we should just revert the change for now…
21:35 <algitbot> [alpine-aports] main/perl: restore the current compatibility - Patchwork: http://patchwork.alpinelinux.org/patch/2961/
22:42 hairyhenderson joined
22:44 Guest45611 joined
23:32 LouisA joined