<    March 2017    >
Su Mo Tu We Th Fr Sa  
          1  2  3  4  
 5  6  7  8  9 10 11  
12 13 14 15 16 17 18  
19 20 21 22 23 24 25  
26 27 28 29 30 31
00:14 daxelrod joined
00:40 dgaff joined
00:44 soveran joined
00:46 <dgaff> Hey all - I'm trying to think of a way to shoehorn a large data structure into a series of hashes and can't quite figure a good way yet - the dataset looks as follows: https://gist.github.com/DGaffney/6d0ae60e803e9fd9df16522402e5734c. Described in human terms, it's the day-by-day listing of how many people move from one subreddit to another, for all of Reddit, for the entire history of reddit up to January 31
00:46 <dgaff> st. So, the data starts in 2005, and is super super small, and gets huge by 2017, so using key as date and subreddit as field, and transitions to that one as value isn't going to work well against hash-max-zipmap-value if I recall correctly? I'm trying to think of how I'd hash all this information and get it nice and even but that's also not leading to any great ideas. Any thoughts?
00:48 daxelrod joined
00:53 <dgaff> Perhaps lists with keys of "{DAY}_{SUBREDDIT}" and vals of the flattened transfers (e.g. ["AskReddit", "100", "AskMeAnything", "50" ...])? The whole dataset in JSON representation is around 23-24GB, the machine has 64GB of RAM for use...
01:19 ScioMin joined
01:31 chipotle joined
01:34 shesek joined
01:43 hahuang65 joined
02:38 lbtxyz joined
02:43 hahuang65 joined
02:46 soveran joined
02:48 rossdm joined
02:50 SirFunk joined
03:00 svm_invictvs joined
03:01 EyePulp joined
03:03 shinnya joined
03:14 hphuoc25 joined
03:14 rchavik joined
03:28 elisiano joined
03:31 bipul joined
03:36 BrianMiller joined
03:44 hahuang65 joined
04:03 Peng joined
04:04 roelmonnens joined
04:35 Cohedrin joined
04:45 hahuang65 joined
04:48 soveran joined
05:08 MrXXIV joined
05:45 hahuang65 joined
06:10 [1]moteus joined
06:13 cyborg-one joined
06:13 hphuoc25 joined
06:16 RemiFedora joined
06:29 kobzar joined
06:29 kobzar left
06:44 bandroidx joined
06:47 hahuang65 joined
06:50 soveran joined
06:54 pila joined
06:57 JimForTheWin joined
07:23 roelmonnens joined
07:31 winteriscoming joined
07:32 hphuoc25 joined
07:42 vchav joined
07:43 winteriscoming joined
07:48 efphe joined
07:48 hahuang65 joined
07:50 SkyRocknRoll joined
07:56 JimForTheWin_ joined
07:57 roelmonn_ joined
08:04 hphuoc25 joined
08:09 daxelrod joined
08:16 hphuoc25 joined
08:18 roelmonnens joined
08:20 BrianMiller joined
08:27 hphuoc25 joined
08:28 soveran joined
08:32 hphuoc25 joined
08:35 Dave_R joined
08:49 hahuang65 joined
08:51 Electron^- joined
09:04 drbobbeaty joined
09:12 rendar joined
09:19 [1]moteus joined
09:50 hahuang65 joined
10:01 sanyo joined
10:04 sanyo1 joined
10:09 hphuoc25 joined
10:15 EyePulp joined
10:18 badboy_ joined
10:29 programmingcool joined
10:44 efphe joined
10:51 hahuang65 joined
10:57 pila joined
11:04 mrwn joined
11:06 drbobbeaty joined
11:22 ogres joined
11:30 sanyo joined
12:00 Siva joined
12:02 <Guest59278> hello.. new to redis .. have a question
12:03 cyborg-one joined
12:05 <Guest59278> anyone there?
12:09 cym3try joined
12:09 cym3try left
12:11 wlightning-fuel joined
12:12 roelmonn_ joined
12:13 <Guest59278> I use php redis .. trying to push data to queue using "RPUSH"
12:13 <Guest59278> it suppose to return "length of the queue"
12:13 <Guest59278> sometimes, when there are high RPS, length of the queue is empty
12:14 <Guest59278> data getting pushed .. sometime not.. rpush is not throwing exceptions ..
12:15 hahuang65 joined
12:32 Mr__Anderson joined
12:34 nathans joined
13:01 tildes joined
13:07 brotspinne joined
13:09 <brotspinne> hi everyone. I want to set up a redis cluster and I'm currently reading all the documentation. what is still not clear to me if I really need sentinel. one thing which is confusing is that the redis cluster has a failure detector (as per https://redis.io/topics/cluster-spec) which does automatic failover to slaves while sentinel does provide the same thing "automatic failover" (as per https://redis.io/topics/sentinel). why use sen
13:11 EyePulp joined
13:16 bauruine joined
13:21 <minus> brotspinne: it's either cluster or sentinel, you cannot use both
13:21 bauruine joined
13:22 <brotspinne> minus: which one should I use?
13:22 <minus> what do you want it to do?
13:22 <minus> failover/HA only? sentinel
13:22 <minus> sharding data? cluster
13:22 <brotspinne> I want redundancy and sharding
13:23 <minus> so you need cluster
13:23 <brotspinne> what is HA?
13:23 <minus> high availability
13:23 <brotspinne> hmm I want high availability and sharding. is this possible?
13:24 <minus> you do get failover with cluster too
13:24 <brotspinne> so redis cluster is a superset of sentinel?
13:24 <minus> it works a bit differently, but in principle, yeah
13:24 <brotspinne> ok this helps a lot
13:27 GreenJello joined
13:28 roelmonnens joined
13:29 <brotspinne> minus: this guy says sentinel isn't for HA either http://stackoverflow.com/a/31143710/616310 I'm a bit confused again
13:30 chipotle joined
13:32 <minus> yeah, it just does failover, your client needs to figure out which of the redis machines to connect to (by asking a sentinel maybe, not sure)
13:39 sanyo joined
13:52 rxc joined
14:09 <brotspinne> minus: do you know if there is a redis python client which supports redis cluster? the recommended one only seems to support sentinel https://github.com/andymccurdy/redis-py
14:10 <brotspinne> or should I just connect to one of the masters and they will negotiate sharding automatically?
14:11 <minus> i've never used cluster, no idea
14:12 <brotspinne> from https://redis.io/topics/cluster-tutorial it seems I should just try to connect to one of the masters and everything else the client should handle automatically but there is so explicit mention about that
14:12 <brotspinne> there is no explicit mention I mean
14:13 Mr__Anderson joined
14:16 hahuang65 joined
14:29 <badboy_> you need a smart client to handle it automatically
14:31 <brotspinne> @badboy_ is there a list of smart clients?
14:31 <brotspinne> this list doesn't mention cluster support for python-py https://redis.io/clients#python
14:31 <badboy_> not that I know of
14:32 <brotspinne> redis-py I mean
14:33 <badboy_> https://github.com/Grokzen/redis-py-cluster
14:34 <brotspinne> hmm this client is not even mentioned in the official list, I would prefer a well known client
14:36 programmingcool joined
14:45 raspado joined
14:59 wlightning-fuel joined
15:05 wlightning-fuel joined
15:17 hahuang65 joined
15:17 wlightning-fuel joined
15:20 al-damiri joined
15:22 daxelrod joined
15:30 <badboy_> lol
15:30 <badboy_> no one added it, that's all
15:30 <badboy_> and afaik it's your best shot
15:30 <badboy_> it also seems active
15:34 Malediction joined
15:35 aapjeisbaas joined
15:39 steeze joined
15:39 ConSi joined
15:39 ScioMin joined
15:42 daxelrod1 joined
15:44 arnaud_ joined
15:55 wlightning-fuel joined
15:55 roelmonn_ joined
16:13 roelmonnens joined
16:14 sz0 joined
16:15 EyePulp joined
16:16 soveran joined
16:16 soveran joined
16:17 hphuoc25 joined
16:17 hahuang65 joined
16:18 wlightning-fuel joined
16:19 wlightning-fuel joined
16:24 WindChimes joined
16:29 SirCmpwn joined
16:30 Malediction joined
16:36 hahuang65 joined
16:36 svm_invictvs joined
16:39 SkyRocknRoll joined
17:05 averythomas joined
17:07 averythomas joined
17:25 Cohedrin joined
17:29 hphuoc25 joined
17:42 efphe joined
17:44 soveran joined
17:54 shesek joined
17:57 SkyRocknRoll joined
17:58 hos7ein joined
18:02 iamchrisf joined
18:03 hahuang65 joined
18:11 hahuang61 joined
18:17 iamchrisf joined
18:26 wlightning-fuel joined
18:34 MrXXIV_ joined
18:37 danns23 joined
18:37 danns23 left
18:46 EyePulp joined
18:46 soveran joined
19:02 cyborg-one joined
19:05 EyePulp joined
19:09 efphe joined
19:17 akkad joined
19:28 timg__ joined
19:28 svm_invictvs joined
19:31 <aapjeisbaas> Anyone in here successfully configured laravel connected to a redis cluster?
19:31 <aapjeisbaas> I use AWS Elasticache with cluter-ON on 3 shard with 2 slaves each.
19:31 <aapjeisbaas> Laravel keeps failing on the MOVED reply of the cluster nodes.
19:31 <aapjeisbaas> lumen.log: https://plakbord.cloud.nl/p/WOmldaAMkkYpgPD04uoWQPmv
19:33 jud^ joined
19:48 gp5st joined
19:49 <gp5st> are the talks at the conference this summer going to be available online?
19:56 Cohedri__ joined
19:58 hahuang65 joined
20:02 sanyo joined
20:03 wlightning-fuel joined
20:04 jud joined
20:06 kulelu88 joined
20:14 forgotmynick joined
20:15 Kronuz joined
20:49 soveran joined
20:51 hos7ein joined
21:01 WindChimes joined
21:01 hashpuppy joined
21:08 WindChimes joined
21:09 wlightning-fuel joined
21:20 Mr__Anderson joined
21:30 iamchrisf joined
21:31 hphuoc25 joined
21:39 ScioMin joined
21:41 atomi joined
21:43 rendar joined
21:43 rendar joined
22:02 roelmonnens joined
22:13 sargas joined
22:38 soveran joined
22:38 soveran joined
22:50 forgotmynick joined
22:56 hahuang65 joined
22:57 roelmonnens joined
23:26 lexAngeles joined