<    April 2017    >
Su Mo Tu We Th Fr Sa  
 2  3  4  5  6  7  8  
 9 10 11 12 13 14 15  
16 17 18 19 20 21 22  
23 24 25 26 27 28 _2_9  
00:00 <kaniini> TemptorSent: that is a maintainer script problem
00:00 <TemptorSent> kaniini: So catching that what was previously defaulted no longer exists would be very useful.
00:00 <kaniini> why God, why me?
00:00 <TemptorSent> kaniini I'm sorry.
00:01 <kaniini> i'm going to go install TempleOS now
00:01 <TemptorSent> I'll leave you in peace.
00:01 <kaniini> God and I need to have a chat
00:01 <TemptorSent> *LOL*
00:02 <kaniini> TemptorSent: i agree that having some sort of trigger to archive previous configuration versions would be desirable
00:02 <TemptorSent> What you are doing is good, no matter what else.
00:02 <TemptorSent> kaniini: Problem solved.
00:02 <kaniini> but we need to worry about minimum viable product before we worry about that
00:03 <TemptorSent> kaniini: As long as you don't go out of your way to make it impossible, I don't see it being much of a hassle at all to be honest.
00:03 <kaniini> http://www.templeos.org/Wb/Home/Web/AppStore/AppStore.html#l1
00:03 <kaniini> it has an App Store
00:03 <kaniini> what more do I need?
00:04 <TemptorSent> kaniini: Filesystems are good -- they can be subverted to just about anything without too much pain :)
00:04 <TemptorSent> Kinda like a religion, right?
00:04 <TemptorSent> ;)
00:05 <kaniini> TemptorSent: he has a hitbox account
00:05 <kaniini> TemptorSent: he spends his entire day coding TempleOS and reading The Drudge Report
00:06 <kaniini> TemptorSent: he does it all from Ubuntu too
00:06 <kaniini> it is quite strange
00:08 <kaniini> TemptorSent: my thing is this
00:09 <kaniini> TemptorSent: shipping something that is 80% of the way there is more valuable than spending time worrying about the remaining 20%
00:09 <kaniini> if i took that approach to things i do, i would never get anything done
00:11 <TemptorSent> kaniini: Understood, I tend to look a couple steps ahead because that's my background.
00:12 <TemptorSent> kaniini: I usually have to deal with full-lifecycle support myself, so I plan ahead accordingly.
00:14 <TemptorSent> kaniini: Different worlds, different balance.
00:28 <kaniini> well for us mere mortals we need to start from a minimum viable product
00:33 <kaniini> TemptorSent: but your input has been quite valuable
00:40 <TemptorSent> kaniini: Usually I deal with products I can't fix once they go out except where I already planned a mechanism, so I don't have much choice but think ahead.
00:40 <TemptorSent> (Because firmware doesn't update by itself :) )
00:42 <TemptorSent> So I try to design so I'm confident that anything I missed is contstrained and can be patched without breaking the rest.
00:42 <TemptorSent> Which is part of why CM is such a big deal for me.
00:43 <TemptorSent> And why I'm almost tempted by Erlang...
00:44 <TemptorSent> (But then I look at the code base!)
01:05 blueness joined
01:49 blueness joined
01:50 s33se joined
02:02 Emperor_Earth joined
02:38 blueness joined
03:45 Guest50859 joined
03:53 TemptorSent joined
03:56 <kaniini> ok
03:56 <kaniini> ok
03:56 <kaniini> TemptorSent: i installed TempleOS
03:56 <kaniini> TemptorSent: have had some chats with God
04:02 <kaniini> TemptorSent: i decided aconf needs to be written in HolyC instead of just C
04:03 <kaniini> TemptorSent: its okay i will include a translator with the source package
04:05 <kaniini> unfortunately God told me systemd is how he intended it to be
04:06 <kaniini> okay i cant continue with this, it is just too ridiculous
04:06 <kaniini> TemptorSent: in all seriousness, do you think using directories to define an entire sub-tree of defaults makes sense?
04:07 <kaniini> TemptorSent: so setup-disk/default/partition0/filesystem = ext4, etc
04:11 ysh joined
04:57 jailbox joined
05:00 mitchty joined
05:12 <asie> trying to rebuild openjdk8 now to see if that fixes it
05:28 ysh joined
06:23 ysh joined
06:24 ysh joined
06:40 czart__ joined
06:45 minimalism joined
07:14 <TemptorSent> kaniini: Just got home from running to the city to get groceries. Did God give you a mandate to cleanse systemd of it's evil before anointing it?
07:14 <TemptorSent> kaniini :)
07:15 <kaniini> no
07:15 <TemptorSent> But realistically, I'm not sure quite how to handle some of the options in a sensical manner.
07:17 <TemptorSent> So if I have /dev/sda2 to be given UUID=12345-67890, have a filesystem of xfs, have a size of 4TB, have a set of mount opts, and be mounted at /var, how do we represent that cleanly?
07:18 <TemptorSent> That's the kind of case I see needing multiple keys as a real possibility.
07:19 <TemptorSent> Now, once you start layering things, it gets even more confusing trying to have it in a directory hierarchy without some sort of consistent reference.
07:20 <TemptorSent> So which order would you specify a luks encrypted raid 6 with ext4 on lvms?
07:21 <TemptorSent> And how do you name things so they're discoverable.
07:21 <TemptorSent> Those are this issues I see needing to be addressed to support the intended functions.
07:22 <TemptorSent> Nothing insurmountable, just need to figure out a scheme and use it consistently.
07:24 <TemptorSent> But while you've got him on the line at the Temple, ask God his opinion :)
07:24 <kaniini> i honestly don't cause it wont boot
07:24 <kaniini> which is probably for the better
07:24 <pickfire> kaniini: Should we build a profile git?
07:24 <pickfire> profiled*
07:25 <TemptorSent> *lol* I'm almost tempted, but I don't have any more machines.
07:25 <pickfire> $ make prefix=/usr profile-fast
07:25 <pickfire> # make prefix=/usr PROFILE=BUILD install
07:25 <pickfire> Alternatively you can run profile feedback only with the git benchmark suite. This runs significantly faster than the full test suite, but has less coverage:
07:25 <TemptorSent> pickfire: It would be interesting to see where it's buring its time, I'm assuming you're talking for optimization?
07:25 <pickfire> TemptorSent: Yeah.
07:26 <Shiz> TemptorSent │ So if I have /dev/sda2 to be given UUID=12345-67890, have a filesystem of xfs, have a size of 4TB, have a set of mount opts, and be mounted at /var, how do we
07:26 <Shiz> │ represent that cleanly?
07:26 <pickfire> Or $ make prefix=/usr profile
07:26 <Shiz> an /etc/fstab with alpine_* prefixed mount options that the installer will then filter out and install into the new system
07:26 <Shiz> ;p
07:26 <Shiz> e.g. alpine_size=4T
07:26 <pickfire> > This may be a good tradeoff for distribution packagers.
07:27 <TemptorSent> pickfire: It's been a long time since I played with profiled-optimizing, but it seemed to be effective mostly on compute bound code. I suspect git has a lot of fs slowdown that would benefit even more from some manual stracing
07:28 <kaniini> i dont think there is any benefit at all to PGO for git
07:28 <kaniini> that screams of lets PGO it because we can
07:28 <TemptorSent> Shiz: Yes, but that doesn't fit kaniini's overall system.
07:28 <pickfire> TemptorSent: Should we do that?
07:28 <kaniini> do what
07:29 <TemptorSent> kaniini: I can think of perhaps a couple chunks of git that would benefit (hashing/tree walking), but not so much the fs calls.
07:29 <kaniini> is there a problem with git's performance ?
07:29 <kaniini> it seems good enough
07:29 <pickfire> kaniini: No
07:29 <TemptorSent> It's not bad, IMHO.
07:29 <kaniini> then why bother with all of this stuff
07:29 <pickfire> Especially with the fat history of aports.
07:29 <kaniini> if it's good enough
07:29 <pickfire> It needs auto gc on every merge.
07:29 <pickfire> T_T
07:30 <TemptorSent> It might be worth benchmarking just to see if you get more than say a 5% speedup.
07:30 <TemptorSent> But if not, don't waste the extra build time.
07:30 <TemptorSent> What's it's time output look like?
07:30 <pickfire> Even 5% is worth it, can be a whole lot faster when we need to git gc for so long.
07:30 <kaniini> i do not need to git gc aports ever
07:31 <pickfire> TemptorSent: Didn't check but probably half a minute?
07:31 <Shiz> i've never ran git gc ever
07:31 <pickfire> kaniini: Because you have fast computer.
07:31 <TemptorSent> pickfire: No, how much time spent in each sys, user, io, block, wait/
07:31 <pickfire> Try using pi as your desktop.
07:31 <kaniini> it's not that fast
07:31 <kaniini> it's only a core i5
07:32 <pickfire> kaniini: Compare core i5 to arm 4 core.
07:32 <pickfire> About 10x faster.
07:32 <TemptorSent> pickfire: Only way to see if there is a win is to give it a shot in a test build and run some benchmarks.
07:32 <pickfire> TemptorSent: How?
07:33 <TemptorSent> pickfire: More than 5% overall is probably worth messing with, but if it's a few percent in some relatively light code, it's not worth the effort.
07:33 <kaniini> and then i am being told by guccifer 3.0 or whatever on #musl how a compiler works
07:33 <kaniini> and it's like
07:33 <pickfire> TemptorSent: Let me test it with git clone aports.
07:33 <pickfire> On a pi 3.
07:33 <kaniini> yeah i dumbed that shit down more than a technically correct description of how it works
07:33 <TemptorSent> pickfire: Compile it stock, take a tarball of a git repo that grinds, (not a clone!)
07:33 <kaniini> but
07:33 <pickfire> But what?
07:33 <kaniini> the point i was making had nothing to do with the compiler itself
07:33 <TemptorSent> Then untar the tarbal to 3 directories.
07:34 <pickfire> Do you want me to test it on a pi 1?
07:34 <Shiz> kaniini: amonakov is a gcc dev iirc, hard to blame him
07:34 <Shiz> :p
07:34 <kaniini> shiz hes probably hacking hillary's emails right now as we speak
07:34 <TemptorSent> *lol*
07:34 <pickfire> haha
07:35 <TemptorSent> pickfire: Anyway, I'd see if you can snag some low hanging fruit, either that, or break out valgrind and oprofile the syscalls (which I suspect are a lot of your burn on gc)
07:35 <kaniini> this is what the US media portrays russian computer science as, how should i know
07:35 <pickfire> Yeah
07:36 <TemptorSent> pickfire: But unless there's a major pain point, I suspect git is not your best use of time for optimization.
07:36 <TemptorSent> Obviously, fixing major pain points is good :)
07:37 <kaniini> on raspberry pi i would look more at memory pressure and disk pressure still
07:37 <TemptorSent> But try to narrow down what's causing the pain using some basic accounting and timing tools before you get into heavy lifting.
07:38 <pickfire> disk io*
07:38 <TemptorSent> Agreed kaniini -- although profiling may be helpful for fixing poor memory usage to some extent at least.
07:38 <pickfire> Yes.
07:38 <pickfire> But I won't look on memory.
07:38 <TemptorSent> pickfire: disk/memory/context switching is going to be the part that hurts.
07:39 <pickfire> So I don't do the profile?
07:39 <TemptorSent> pickfire: Also, bad cache assumptions may be an issue in git, since it was designed on x86.
07:39 <pickfire> I thought distributions should pack profiled packages?
07:39 <TemptorSent> pickfire: Start with the basics before you profile it -- that will help you decide which profiling options to use.
07:40 <pickfire> ?
07:40 <TemptorSent> pickfire: Not necessarily always worth the effort.
07:40 <pickfire> Oh, okay.
07:40 <TemptorSent> pickfire: If you're io bound and it's the io scheduler that's eating you alive, fixing git isn't going to help.
07:41 <kaniini> if youre i/o bound on a raspberry pi
07:41 <TemptorSent> pickfire: So figure out which resource is getting consumed fastest and address that first.
07:41 <kaniini> its probably because youre using the onboard NOR flash or you have a really shitty SD card
07:41 <kaniini> tbh
07:42 <kaniini> if the former, don't worry about it because you will need to replace your raspberry pi soon most likely
07:42 <kaniini> if the latter, just buy a better SD card
07:42 <TemptorSent> kaniini: Their usb bus is pretty crappy, so if you're running more than one device (wireless included) it can quickly slow down.
07:42 <pickfire> What about laptop?
07:42 <pickfire> I mean I still need gc on laptop as well.
07:43 <TemptorSent> pickfire: If you have real disk controllers, your memory starts helping a lot more with caching.
07:43 <TemptorSent> pickfire: Right, but what's the slow part? random seeks? sequential reads? cache misses?
07:43 <pickfire> Not sure.
07:43 <pickfire> Didn't trace it.
07:44 <TemptorSent> pickfire: Read/modify/write cycles are notoriously bad for throughput for instance.
07:44 <TemptorSent> pickfire: I'd start with time/top with a couple workloads just to figure out what's taking the time.
07:45 <pickfire> time can do that?
07:45 <pickfire> I mean time and top can't measure io.
07:45 <TemptorSent> pickfire: Then try running some basic memory/disk/io benchmarks depending on what you get.
07:45 <TemptorSent> pickfire they'll tell you how long it's blocked.
07:46 <pickfire> TemptorSent: Nevermind, I guess need not to do it then.
07:46 <TemptorSent> pickfire: It might be worth while if you have a high user time compared to real and sys
07:47 <pickfire> What do you mean?
07:47 <TemptorSent> or if sys is high and it's all from one syscall loop.
07:47 <TemptorSent> time git gc and see what the times for user, sys, and real are.
07:47 <pickfire> TemptorSent: I don't know what is the difference between real, sys and user time.
07:47 <TemptorSent> if it takes 20mins real, 4 mins user, and 2 mins sys, it's probably io bound, not cpu.
07:48 <TemptorSent> if you have high sys, it could be io or memory bound
07:48 <kaniini> or it is badly written code
07:48 <TemptorSent> typically, compute bound is a high user time with respect to wall clock.
07:48 <pickfire> Ah
07:48 <kaniini> for example epoll_wait(3) loop that never ACKs socket errors
07:48 <TemptorSent> kaniini: Yes, that could be the cause.. and frequently is!
07:48 <kaniini> will burn 100% cpu
07:49 <kaniini> as sys
07:49 <TemptorSent> Yeah, if sys looks way out of line, it's probably something like that ^^^
07:49 <pickfire> What about high user time?
07:50 <TemptorSent> high user time is usually a good indication of cpu bound workloads, once you check you're not hitting the memory wall (top is good for that)
07:51 <TemptorSent> so if you see 70% of the real time is user, profiling may be worth while.
07:51 <TemptorSent> pickfire: It just lets you know which tree to bark up when you're looking for the problem.
07:51 <kaniini> i don't like the security aspects of PGO
07:51 <kaniini> the whole concept seems sketchy to me
07:51 <TemptorSent> pickfire: high user, low sys, and generally slow often indicates bad loop usage.
07:52 <pickfire> Ah
07:52 <pickfire> TemptorSent: Nice, thanks a lot for telling me that.
07:52 <pickfire> I learn something.
07:52 <TemptorSent> kaniini: I spent some time auditing some profile-optimized binaries I did many years ago and they were pretty clean.
07:53 <TemptorSent> kaniini: It basically was just generating the branch trees proability and using that to inform the optimizer.
07:53 <kaniini> that part is fine, but it is not really reproducible on a builder
07:53 <kaniini> i mean stuff like what JVM does
07:54 <TemptorSent> kaniini: So it would unroll loops more agressively where profiling indicated it was spinning, re-order branch instructions to put the most frequent first, etc.
07:55 <TemptorSent> kaniini: I haven't looked at the latest developments, but I wouldn't be surprised to see a previously usefull tool broken beyond use.
07:55 <kaniini> i rather just fix the code
07:55 <TemptorSent> kaniini: The biggest value I found in using it was identifying areas to optimize in the code paths.
07:56 <kaniini> than deal with any of that
07:57 <pickfire> TemptorSent, kaniini: http://ix.io/ptm
07:57 <TemptorSent> kaniini: That's what I was using it for, profile and benchmark with a matrix of conditions, then determine which one gives the best performance in which case, and include all of the above, with a code-path optimizier in line
07:57 <pickfire> On my laptop.
07:57 <pickfire> i7
07:57 <kaniini> ok that is great and all but it doesnt change that we cant really do a PGO build on the builders
07:57 <kaniini> as it is not reproducible
07:58 blueness joined
07:58 <TemptorSent> kaniini: Agreed.
07:58 <pickfire> I really hope github accept shallow clone push.
07:58 <pickfire> Shitty github.
07:58 <TemptorSent> kaniini: If it could identify some places to put #pragmas in the git code to speed it up, it might be worth the effort.
07:59 <pickfire> If they just accept it, I just clone --depth 10
07:59 <TemptorSent> At 9 mings user, you'll want to do something like an iostat next
07:59 <pickfire> Or even better, just submit patches to ML to avoid this.
07:59 <kaniini> lol
08:00 <TemptorSent> But that's enough to be possibly interesting if it's not showing that's all disk grind.
08:00 <pickfire> TemptorSent: Might not be io bound.
08:00 <pickfire> I use raid 0 ssd.
08:01 <kaniini> on raspberry pi?
08:01 <TemptorSent> pickfire; Looking at the low sys, it's not looking like it's spinning in syscalls, and the real isn't indicating blocking badly.
08:01 <kaniini> ;)
08:01 <pickfire> Well, 1 disk is encrypted.
08:01 <pickfire> kaniini: On x220 i7 8gb ram
08:02 <kaniini> oh well i am going to bed now
08:02 <kaniini> goodnight
08:03 <pickfire> Night.
08:04 <pickfire> TemptorSent: Probably cpu bound 16319 ivan 20 0 597.2m 547.9m 388.1 6.9 2:27.03 S `- git
08:04 <pickfire> disk on iostat shows 5%
08:04 <TemptorSent> Goodnight kaniini.
08:05 <pickfire> TemptorSent: What do I do for this case?
08:05 <TemptorSent> pickfire: Okay, that might be worth profiling then.
08:05 <pickfire> 9 min but not 30 sec
08:05 <pickfire> T_T
08:05 <TemptorSent> pickfire: Like I said, good to eliminate as much as possible before banging your head against something more complex.
08:06 <TemptorSent> pickfire: If I had to guess, it's probably in a copy and hashing tree loop.
08:07 <TemptorSent> pickfire: I'm not sure how it's actually implemented in git, probably a red/black tree.
08:08 <pickfire> Huh?
08:08 <pickfire> What's that?
08:08 <TemptorSent> pickfire: Basically a self-balancing binary tree
08:08 <pickfire> Ah
08:08 <pickfire> Haven't heard of red/black tree.
08:08 <pickfire> I don't have much knowledge about self-balancing binary tree.
08:09 <TemptorSent> pickfire: They're very common in binary search and hash trees.
08:09 <pickfire> Ah
08:10 <TemptorSent> It's just a b-tree that enforces a more or less equal branch structure on both sides
08:11 <pickfire> Ah
08:11 <TemptorSent> Basically, the way it works is by not growing any deeper on a given branch until that is the minimum branch depth.
08:11 <TemptorSent> (actually, 1 link deeper maximum)
08:13 <TemptorSent> So you count the number of black nodes you pass through to get to your leaf node, and only add another level once all black nodes have the same number of traversals to reach a leaf.
08:13 <pickfire> Ah
08:13 <pickfire> So basically to make the level stay the same.
08:14 <pickfire> TemptorSent: And git gc does the rebalancing?
08:14 <TemptorSent> So the maximum difference in longest distance is basically a 2:1 ratio
08:14 <TemptorSent> It constrains your traversals and keeps your tree from getting lopsided.
08:15 <pickfire> Ah
08:15 <pickfire> TemptorSent: Can you please tell me more about using tools like time, top and iostat.
08:15 <pickfire> ?
08:16 <TemptorSent> pickfire: Nothing really astounding about them, they just give you different views into what resources are being used how (free being another)
08:16 <pickfire> TemptorSent: Actually, the build is fast, the test is slow.
08:16 <pickfire> Ah
08:16 <TemptorSent> top is very useful for seeing when you have a sudden spike in activity in real time.
08:16 <pickfire> I don't quite understand those avg-cpu: %user %nice %system %iowait %steal %idle
08:17 <pickfire> don't know what is the difference.
08:17 <TemptorSent> iostat will give you a good idea of how hard you're hitting the disk vs. the rest of the process.
08:17 <pickfire> Ah
08:18 <pickfire> Vs. the rest of the procell?
08:18 <pickfire> process*
08:18 <TemptorSent> %user is percent of time running userspace, %system is syscalls mostly, %iowait is blocking %idle is spinning without resource
08:18 <pickfire> spinning without resource?
08:19 <TemptorSent> %steal is steal-time accounting I believe, and %nice is related to it's nice leve?
08:19 <TemptorSent> pickfire: In an idle loop, not burning cpu.
08:19 <pickfire> Ah
08:19 <pickfire> Just burning io?
08:19 <TemptorSent> pickfire: Waiting for a callback, like a server sitting with nothign to do.
08:19 <pickfire> Ah
08:20 <TemptorSent> pickfire: No, not using any resources, just spinning idle.
08:20 tty joined
08:21 <pickfire> Okay.
08:21 <TemptorSent> pickfire: Basically, once a service is up and running, it doesn't need to do much of anything until it gets a connection, so it uses something like while(1){ wait_connection(); }
08:22 <pickfire> Oh
08:22 <TemptorSent> pickfire: Or something more useful :)
08:22 <pickfire> TemptorSent: Does stealing means that the process steal resources from another process?
08:23 <TemptorSent> pickfire: It has to do with accounting more...
08:23 <pickfire> Woad
08:23 <TemptorSent> I don't really use it.
08:24 <TemptorSent> It's basically for vms with overprovisioning of allocated resources.
08:25 <pickfire> Ah
08:27 <TemptorSent> It's a measure of how much of your allocated cpu time you're using.
08:28 <TemptorSent> If you're not getting as much cpu time as you request, your steal time goes up IIRC
08:28 <pickfire> Oh
08:29 <TemptorSent> pickfire: It's something I don't usually worry too much about on my workloads unless something is acting haywire.
08:30 <pickfire> TemptorSent: How do you normally choose file system and io scheduler?
08:30 <TemptorSent> I'd have to check docs for the exact symantics.
08:31 <TemptorSent> pickfire: By my intended workload mostly.
08:31 <pickfire> docs?
08:31 <TemptorSent> pickfire: kernel docs on 'Steal time accounting'
08:31 <TemptorSent> pickfire: /usr/src/linux/Documentation
08:32 <TemptorSent> or kernel.org/doc/Documentation
08:33 <pickfire> /usr/src?
08:33 <TemptorSent> pickfire: That's where the linux src lives when you extract it.
08:33 <pickfire> Ah
08:34 <TemptorSent> pickfire; Beyond that, UTS,L.
08:34 <pickfire> UTS,L.?
08:35 <TemptorSent> Use The Source, Luke.
08:36 <TemptorSent> pickfire: See the Jargon file for other common (and not so common these days) abbreviations.
08:36 <pickfire> Aha
08:37 <pickfire> TemptorSent: Luke Skywalker Use The Force. ^^
08:37 <TemptorSent> Things you're come across in unix-land at random.
08:37 <TemptorSent> pickfire: Yes, it's a bad pun on that.
08:37 <pickfire> I haven't came across that.
08:38 <TemptorSent> BOFH being one of the more common ones. Got the pin.
08:39 <TemptorSent> But back to the original question of profiling, now that you know what part is taking all the time, you can choose your profiling flags appropriately.
08:40 <TemptorSent> pickfire: The syscalls don't seem to be your issue, so it's going to be loop unrolling, branching, and caching in the copy/hash/index loop that you'll be looking at.
08:40 <pickfire> BOFH?
08:40 <TemptorSent> Look that one up ;)
08:41 <pickfire> TemptorSent: Wow, you can even deduce that from the output of `time`. Interesting.
08:41 <pickfire> Haha, Bastard Operator From Hell.
08:42 <TemptorSent> pickfire: Well, the output of time, iostat, and the general design of git anyway.
08:42 blueness joined
08:42 <TemptorSent> :)
08:43 <TemptorSent> GC is basically walking the tree, so you've got a set of loops that read, hash, copy, compare, and copy again IIRC.
08:44 <TemptorSent> And some time spent in rehash/reindex
08:44 <TemptorSent> It's just the nature of DAGs, really.
08:45 <TemptorSent> (And Merkle-DAGs/block-chains)
08:45 <TemptorSent> (And please excuse any horribly butchered spellings)
08:46 blueness joined
08:49 <* pickfire> found too many jargons
08:49 <pickfire> Too deep, can't understand much.
08:49 <TemptorSent> DAGs are basic structures in computing - Directed Acyclic Graphs
08:49 <pickfire> Ah
08:49 <pickfire> I think I have seen something similar.
08:50 <TemptorSent> Merkle DAGs are a class of hash-based DAGs
08:50 <pickfire> acyclic graph traversal
08:50 <pickfire> Found that in http://pkgconf.org/features.html
08:50 <TemptorSent> Yes, exactly.
08:50 <pickfire> TemptorSent: But don't know what is DAG.
08:51 <TemptorSent> Directed simply means that you have a link from a parent to a child node.
08:51 <TemptorSent> Acyclic means that it can't loop back on itself
08:51 <pickfire> Ah
08:51 <pickfire> A never ending loop
08:52 <TemptorSent> And Graph implies that the structure is represnted by nodes (usually containing the data), and edges, which form the links, but generally only carry meta-data if anything.
08:53 <TemptorSent> So a DAG is a set of nodes that has traversal patterns making a tree struture from the top down, but no cross links or reverse linking.
08:54 <pickfire> Like a -> b -> c -> d
08:54 <TemptorSent> Then what git does is uses those hashes as keys in an index pointing to the actual data in the pack files.
08:54 <TemptorSent> pickfire Yes.
08:55 <TemptorSent> d<-c<-a->b->e as well :)
08:56 <pickfire> TemptorSent: Does apk uses topological sorting?
08:56 <TemptorSent> So what gc does is traverses the tree, scans the index, scans the pack file, and prunes any orphans, then compresses and rebalances I believe.
08:57 <TemptorSent> pickfire: I believe so? I didnt' get into it in depth yet.
08:57 <TemptorSent> pickfire: And I'm also far from an expert on git's internals, I just know the basics.
08:59 <pickfire> Ah
08:59 <TemptorSent> pickfire: Hopefully that helps you approach profiling optimizations with a little better understanding?
08:59 <pickfire> Yeah.
09:00 <TemptorSent> And gives you an idea of what's likely to be in your hot loop.
09:00 <TemptorSent> Cache misses there can really drag things down too.
09:01 <TemptorSent> But I suspect that it's mostly going to turn out that git does a lot of work :)
09:02 <TemptorSent> See if you can knock 5-10% off the top of your runtime with a profiled build -- if so, twiddle profile options until you figure out which one or combination is making the major impact, then try to fix the code-path in the source if possible.
09:04 <TemptorSent> But really optimizing git is going to be a much bigger project than just running profile guided optimizations -- I believe it was written without using any of the matrix math functions of modern cpus...
09:04 <TemptorSent> Find someone that's good at that kind of work take a crack at it if you want to see big speedups.
09:05 <pickfire> wow
09:05 <TemptorSent> pickfire: Just remember the law of diminishing returns
09:06 <pickfire> But probably not that fast.
09:06 <TemptorSent> pickfire: It all depends on your workload and needs.
09:09 fekepp joined
09:28 <TemptorSent> Alright, I'm off to bed. Goodnight pickfire.
09:47 medber joined
09:53 blueness joined
10:04 t0mmy joined
10:09 <pickfire> Night.
12:04 schndr_ joined
12:16 fabled_ joined
12:21 medber joined
12:38 fekepp joined
14:15 minimalism joined
16:10 ostera joined
16:24 schndr_ joined
17:56 ostera joined
18:00 chris| joined
18:07 DLange joined
18:33 <TemptorSent> So, it turns out that MirBSD's pax doesn't even know how to handle pax archives!
18:34 <TemptorSent> Does anyone have schilytools packaged up somewhere perhaps?
19:39 <asie> Error relocating /usr/lib/libglib-2.0.so.0: pthread_setname_np: symbol not found
19:39 <asie> after today's update
19:54 algitbot joined
20:06 <ashb> Okay I'm getting hunderes of these a day. 'segfault at 0 ip 00007f7b101c8908 sp 00007ffd76bad118 error 4 in ld-musl-x86_64.so.1[7f7b1017a000+88000]'
20:06 <ashb> How would I go about debugging it?
20:29 schndr_ joined
20:39 <kaniini> find the process gdb it
20:59 <ashb> It's an exim process. child I guess as nothing is showing up in the exim logs. Odly.
21:05 schndr_ joined
22:33 <mitchty> question for getting https://github.com/alpinelinux/aports/pull/684 into testing, i'm updating that port to 1.0 idris (yes they released 1.0 on april first)
22:34 <mitchty> would the fact that the build stage uses cabal to download the dependencies invalidate it from getting into testing?
22:35 <mitchty> rather, what approach for things built via ghc dependencies should be taken, to note, there are around 100 total transitive dependencies for this compiler
22:37 blueness joined