<    April 2017    >
Su Mo Tu We Th Fr Sa  
 2  3  4  5  6  7  8  
 9 10 11 12 13 14 15  
16 17 18 19 20 21 22  
23 24 25 26 27 28 29  
00:25 svm_invictvs joined
00:27 trevor joined
00:29 <sumobob> just finished witha pretty complicated aggregation query
00:29 <sumobob> blown away by the power of mongo
00:29 <sumobob> there are so many options, I was hoping someone might be able to take a look at my query here: https://gist.github.com/anonymous/366df7cafa2fd561dee599af6f701121
00:29 <sumobob> and let me know what they thing
00:29 <sumobob> think*
00:29 culthero joined
00:32 <fmt_caret> sumobob: just be cognizant of the fact that aggregations eat into your database’s CPU, particularly as the amount of data being aggregated increases and the operations are complex
00:32 <sumobob> yeah I have this running on a cron job every midnight
00:33 <sumobob> I'm saving the results in a seperate collection, then just loading those when needed
00:33 <sumobob> I'm wondering if theres a way to get rid of that second $group
00:34 <sumobob> i basically have an array like [{sent: 100, received: 0 }, { sent: 0, received: 100}] before that stage, and I'm just summing them together to get {sent: 100, received: 100 }
00:40 fmt_caret joined
00:48 Gwayne joined
00:51 kyuwonchoi joined
00:52 re1_ joined
01:03 joshontheweb joined
01:25 StephenLynx joined
01:28 ksa joined
01:28 mbwe joined
01:28 Ratler joined
01:29 omnidan joined
01:31 fmt_caret joined
01:40 artok joined
01:59 druonysus joined
02:00 kyuwonchoi joined
02:03 sumobob joined
02:04 synchroack_ joined
02:31 freddy___ joined
02:35 NoOutlet joined
02:45 sumobob joined
02:58 fels joined
03:00 orbyt_ joined
03:08 kyuwonchoi joined
03:15 siruf joined
03:22 artok joined
04:00 kt2 joined
04:12 druonysus left
04:16 kyuwonchoi joined
04:29 sumobob joined
04:39 freddy___ joined
04:41 ayogi joined
04:42 fredd____ joined
04:53 DyanneNova joined
05:04 fmt_caret joined
05:16 lpin joined
05:16 sumobob joined
05:25 kyuwonchoi joined
05:34 preludedrew joined
05:53 svm_invictvs joined
05:59 freddy___ joined
06:03 evil_gordita joined
06:06 coudenysj joined
06:32 kyuwonchoi joined
06:37 coudenysj joined
06:42 castlelore joined
07:07 jri joined
07:17 fels joined
07:26 artok joined
07:29 freddy___ joined
07:36 Lujeni joined
07:40 kyuwonchoi joined
07:43 fels joined
07:49 timg__ joined
07:50 quattro_ joined
07:54 rendar joined
07:54 mylord joined
08:26 aV_V joined
08:27 artok joined
08:27 coudenysj1 joined
08:29 coudenysj2 joined
08:29 jwd joined
08:36 gfidente joined
08:47 kyuwonchoi joined
08:53 coffeejunk joined
09:10 freddy___ joined
09:19 fredd____ joined
09:27 artok joined
09:28 mylord joined
09:34 undertuga joined
09:51 jri joined
09:57 kyuwonchoi joined
09:57 kyuwonchoi joined
10:00 timg__ joined
10:13 kyuwonchoi joined
10:13 kyuwonchoi joined
10:18 VeeWee joined
10:21 ssarah joined
10:28 artok joined
10:42 kexmex joined
10:48 pagios joined
10:48 <pagios> hola, what is the equiv to union select in nosql?
10:50 <Derick> that's a very generic question
10:50 <Derick> in MongoDB, you can probably do this through the aggregation framework
10:52 <Derick> but a union is that not just an $or query?
10:56 mylord joined
11:03 <pagios> Derick, thanks
11:03 <pagios> can i use regular expressions in mongoDB?
11:04 <pagios> db.logins.find({ username: { $ne: 1 <regular expressions here> } })
11:11 <pagios> resolved thanks :)
11:11 pagios left
11:13 harry1 joined
11:14 kyuwonchoi joined
11:50 dump joined
12:02 jamesaxl joined
12:06 yeitijem joined
12:07 alexi5_wrk joined
12:09 alexi5_wrk left
12:09 alexi5_2 joined
12:09 artok joined
12:10 alexi5_wrk joined
12:11 alexi5_wrk joined
12:24 jita joined
12:24 <jita> In this structure I have millions of items http://dpaste.com/09BTBGZ. Should I use this structure to model my data or use something else? Suggestions are welcome please
12:25 <jita> by millions of items I mean in the array
12:27 verwilst joined
12:31 <jita> anyone please ?
12:31 <jita> I want to optimize for filtering on the items as well
12:38 silenced joined
12:46 jwd joined
12:52 FatNoLife joined
12:52 re1 joined
12:52 FatNoLife joined
12:54 Derick joined
12:55 Derick joined
12:57 joshontheweb joined
13:01 ramortegui joined
13:09 sterns joined
13:22 Siegfried joined
13:30 jwd joined
13:38 mylord joined
13:45 artok joined
13:47 kexmex joined
13:56 fmt_caret joined
14:00 geoffb joined
14:17 culthero joined
14:19 culthero joined
14:23 firebalrog joined
14:27 fmt_caret joined
14:28 DyanneNova joined
14:39 beawesomeinstead joined
14:48 freddy___ joined
14:49 mkoistinen joined
15:02 _MPZ joined
15:02 hays_ joined
15:02 hays_ joined
15:03 necrogami_ joined
15:03 dustinm`_ joined
15:03 SteamWells_ joined
15:04 ams___ joined
15:04 anonfriese joined
15:04 ggherdov`_ joined
15:05 jefferai_ joined
15:05 supershabam_ joined
15:06 mrbro_ joined
15:06 fels joined
15:07 Necro|senseless joined
15:09 maxmanders_ joined
15:09 hayer_ joined
15:09 jeansch_ joined
15:10 jokke1 joined
15:11 RunThroughMedia_ joined
15:11 firebalrog joined
15:11 RunThroughMedia_ joined
15:13 StephenLynx joined
15:13 jri joined
15:16 SpeakerToMeat joined
15:16 gentunian joined
15:20 fmt_caret joined
15:21 davidmichaelkarr joined
15:25 fuzzybear3965 joined
15:26 arnas joined
15:26 <arnas> hello
15:26 <arnas> I have User schema and it has an array of lesson objects. Each object has an id. How can I find a lesson in that array by an id?
15:27 <fuzzybear3965> Can you give us an example of a document?
15:27 blizzow joined
15:27 <fuzzybear3965> It helps to have an example of a typical document.
15:28 <arnas> fuzzybear3965, document as in my user schema structure?
15:28 <fuzzybear3965> jYes.
15:28 <fuzzybear3965> *Yes.
15:28 <fuzzybear3965> Either report your schema (including type of all values) or an example document.
15:29 <arnas> fuzzybear3965, is a screenshot ok?
15:29 <fuzzybear3965> arnas, just copy-paste the object.
15:29 <fuzzybear3965> Sure, whateve.
15:29 <fuzzybear3965> screenshot is fine.
15:30 <arnas> fuzzybear3965, https://www.dropbox.com/s/9p0i6wlbc3479yc/Screenshot%202017-04-13%2018.30.02.png?dl=0
15:30 <fuzzybear3965> What do you want to query?
15:31 <fuzzybear3965> arnas?
15:31 <arnas> fuzzybear3965, I need to get an object of startedLessonsInfo by it's id
15:31 <fuzzybear3965> Which id?
15:32 <fuzzybear3965> The _id id or the startedLessonsInfo.0.id id?
15:32 <arnas> fuzzybear3965, I have an user id and lesson id
15:32 <arnas> fuzzybear3965, for user _id and for startedLessonsInfo id
15:32 <arnas> like... that object id
15:32 <fuzzybear3965> Okay.
15:32 <fuzzybear3965> You want find documents that match a lesson id?
15:33 <arnas> fuzzybear3965, yes, but for a user that has uerId
15:33 <arnas> userId*
15:33 <fuzzybear3965> There is no userId in your example.
15:33 <arnas> fuzzybear3965, _id
15:33 <fuzzybear3965> Those are different things.
15:33 <arnas> sorry
15:34 <fuzzybear3965> Do you mean a field "userid" or the field "_id"?
15:34 <arnas> _id
15:35 <fuzzybear3965> Do you know, at query time, the startedLessonsInfo element you want to access?
15:35 <fuzzybear3965> i.e. startedLessonsInfo[0] or startedLessonsInfo[1] or ..?
15:35 <arnas> fuzzybear3965, no, I only know id
15:35 <fuzzybear3965> Which id?!
15:35 <fuzzybear3965> Both?
15:35 <arnas> user id and started lessons info id
15:36 jwd joined
15:36 <arnas> fuzzybear3965, I am not sure how else can I name them
15:36 svm_invictvs joined
15:36 <fuzzybear3965> You specify data by its location in the document.
15:36 <fuzzybear3965> _id is the document's unique identifier.
15:36 <arnas> fuzzybear3965, so I can't search by _id? is that what you meant? o.O
15:36 <fuzzybear3965> startedLessonsInfo[lesson].id is the lesson id.
15:37 <fuzzybear3965> You can definitely "query" by _id.
15:37 <fuzzybear3965> I'm just confused about what information you *have to form your query* and what information you *want to obtain*.
15:37 <fuzzybear3965> It seems like you the value of _id and startedLessonsInfo[lesson].id at query time.
15:37 <arnas> fuzzybear3965, ok, I have userId and lessonId and I need to update an object in startedLessonsInfo
15:38 <fuzzybear3965> THERE IS NO userId field!!!
15:38 <fuzzybear3965> It doesn't exist.
15:38 <arnas> I know, that's how I am calling _id
15:38 <fuzzybear3965> There is no lessonId field!!!
15:38 <arnas> just to be more clear
15:38 <fuzzybear3965> Don't!
15:38 <fuzzybear3965> It's less clear.
15:38 <arnas> ok, so I have _id and id
15:38 <fuzzybear3965> My specification is clear.
15:38 <fuzzybear3965> That's how you should do it.
15:38 <fuzzybear3965> No, you don't.
15:38 <fuzzybear3965> There is no id field.
15:38 <arnas> there is?
15:38 <fuzzybear3965> There is a startedLessonsInfo[lesson].id field for every lesson in the startedLessonsInfo array.
15:39 <fuzzybear3965> No, there's not.
15:39 <arnas> so how can I find it?
15:39 <fuzzybear3965> I don't see any root level id field.
15:39 <fuzzybear3965> Do you see one?
15:39 <arnas> I didn't say it was root
15:39 <fuzzybear3965> You did.
15:39 <fuzzybear3965> Saying "I have the value of the id field" dictates there is a root level id field.
15:39 <fuzzybear3965> If you don't specify a path to the id field then, by default, it's root level.
15:39 <arnas> I see
15:40 <fuzzybear3965> So, to clarify...
15:40 <fuzzybear3965> You have a value for the _id field and a value for the startedLessonsInfo[lesson].id field.
15:40 <arnas> yup
15:40 <fuzzybear3965> You want to know which documents match that _id and that startedLessonsInfo[lesson].id.
15:41 <fuzzybear3965> First, you should query over _id. That'll return the document that matches that id.
15:41 <fuzzybear3965> *that _id.
15:41 <arnas> okay, that one is pretty easy
15:41 <fuzzybear3965> Then, with the returned document you should loop over the startedLessonsInfo array, looking for matches on the startedLessonsInfo[loop_idx].id field.
15:42 <arnas> fuzzybear3965, that's the problem. I don't just need the data from startedLessonsInfo, I need to update a specific object from that array
15:44 <fuzzybear3965> Which object? The entire startedLessonsInfo array or a particular sub-object of that array?
15:44 <arnas> fuzzybear3965, startedLessonsInfo[lesson].id object
15:44 <arnas> fuzzybear3965, so.. particular sub-object of that array
15:44 <fuzzybear3965> That's not an object. That's just a key-value pair.
15:45 <arnas> fuzzybear3965, well, it's pointing to that sub-object
15:45 <fuzzybear3965> startedLessonsInfo[lesson].id isn't an object is my point.
15:45 <fuzzybear3965> But, that doesn't matter too much.
15:45 <arnas> fuzzybear3965, well, I suppose you get the point
15:47 <fuzzybear3965> Yeah, okay. So, you only want to update particular elements of the startedLessonsInfo array based on that array element containing a particular value of startedLessonsInfo[element].id?
15:47 silenced joined
15:47 <arnas> element*, but yes
15:47 <fuzzybear3965> Guaranteed to be only one?
15:47 <arnas> fuzzybear3965, well, the id is always unique, isn't it?
15:48 <fuzzybear3965> Which id?
15:48 <fuzzybear3965> The _id id or the startedLessonsInfo[lesson].id id?
15:48 <arnas> startedLessonsInfo[lesson].id id
15:48 <arnas> well, actually both
15:48 <fuzzybear3965> No.
15:48 <arnas> o.O how so?
15:48 <fuzzybear3965> The user specifies startedLessonsInfo[lesson].id.
15:48 <fuzzybear3965> mongoDB restricts the _id value to be unique to a document.
15:48 <arnas> huh
15:49 <arnas> but it's ObjectId
15:49 <fuzzybear3965> ^^^^
15:49 <fuzzybear3965> Everything besides _id is specified by the user.
15:49 <fuzzybear3965> *is typically specified
15:50 <fuzzybear3965> mongoDB disallows any two documents in a collection from sharing the same _id value.
15:50 <arnas> fuzzybear3965, well, I didn't specify startedLessonsInfo[lesson].id id, that's for sure
15:51 <fuzzybear3965> It's bad practice to have multiple ObjectIDs in a document.
15:51 <fuzzybear3965> The purpose of the ObjectID is to uniquely specify a document.
15:51 <Derick> hmm
15:51 <arnas> fuzzybear3965, I am pretty sure it inserted it automatically
15:51 <Derick> arnas: by your ODM perhaps
15:51 <arnas> Derick, huh?
15:51 <Derick> fuzzybear3965: that's not the purpose of an ObjectID
15:52 <fuzzybear3965> If you encounter a situation where you have this then it's advisable to restructure your data so that you have one ObjectID per document.
15:52 <Derick> fuzzybear3965: the purpose of the _id field is that it uniquely specifies a document - you don't need to use ObjectIDs for it, and you can use ObjectIDs to (for example) refer to other documents
15:52 <fuzzybear3965> That's true, you don't need to use ObjectIDs.
15:52 <Derick> arnas: MongoDB does not add data itself (besides the _id field in a document if not set)
15:52 <fuzzybear3965> But, they are used by default.
15:53 <arnas> Derick, I am sure as hell didn't add that myself
15:53 <fuzzybear3965> arnas, when you run your .findAndUpdateOne() you're going to receive an object, which you should be able to modify in the callback.
15:54 <arnas> fuzzybear3965, so I need to search just by _id?
15:54 <fuzzybear3965> I might be wrong, but that would be my first attempt.
15:54 re1_ joined
15:55 RunThroughMedia_ joined
15:56 <fuzzybear3965> The problem is, you don't seem to know a priori, which element of startedLessonsInfo you need to modify at query time.
15:56 <fuzzybear3965> I would strongly recommend restructuring your data so that your query will return exactly that which you want to update.
15:57 <arnas> fuzzybear3965, I am not sure how I should restruct it
15:58 <fuzzybear3965> arnas, just search by _id and try modifying the data in the callback and inspect the document to see if it was modified post hoc.
15:59 <fuzzybear3965> arnas, I'm not sure, either. That depends on your use case.
15:59 <arnas> fuzzybear3965, ok, let me try that
16:03 <arnas> fuzzybear3965, not, didn't work :((
16:04 <arnas> fuzzybear3965, what if I make a copy of startedLessonsInfo, modify what I need and then run a query to update the whole array?
16:04 <fuzzybear3965> Okay, then you need to know the element of the array that you need to update a priori.
16:04 <fuzzybear3965> Do you have that?
16:04 <arnas> no :(
16:04 <fuzzybear3965> That would work, conditional upon you knowing which parts of the array you need to modify and how you need to modify them.
16:05 <arnas> yeah, I do know that
16:06 <fuzzybear3965> Okay, then, yes.
16:12 timg__ joined
16:12 <arnas> fuzzybear3965, ok, I'll try to do that then
16:12 jri joined
16:16 harttho joined
16:18 poz2k4444 joined
16:22 m0nument joined
16:25 synchroack_ joined
16:26 silenced joined
16:28 orbyt_ joined
16:28 arnas_ joined
16:31 jefferai_ joined
16:31 necrogami_ joined
16:31 ggherdov`_ joined
16:31 LouisT_ joined
16:32 blizzow joined
16:32 sppadic_ joined
16:33 m1dnight_ joined
16:39 silenced joined
16:39 jri joined
16:43 mkoistinen joined
16:47 ggherdov` joined
16:49 mkoistinen joined
16:50 m0nument left
16:54 re1 joined
16:57 jri joined
16:59 quattro_ joined
17:02 jri_ joined
17:03 siddc joined
17:04 <siddc> Hi, we are planning on running some Mongo hosts in AWS. These DBs will be read-only no writes. Is there any way to put the data directory of the Mongos on a shared drive so that we can scale the Mongos easily since they will also have a loaded dataset?
17:07 fiQmob joined
17:08 jri joined
17:16 timg__ joined
17:18 jaequery joined
17:19 m0nument joined
17:24 erkules joined
17:32 arnsa joined
17:33 lkannan joined
17:37 fels joined
17:37 fels joined
17:50 erkules left
17:52 <m0nument> screen -r
18:02 fels joined
18:07 ionelmc joined
18:08 fmt_caret joined
18:10 Sasazuka joined
18:10 SkyRocknRoll joined
18:10 SkyRocknRoll joined
18:20 sadmac joined
18:28 philipballew joined
18:28 poz2k4444 joined
18:47 sadmac left
18:58 arnsa joined
19:08 jaequery joined
19:10 re1_ joined
19:10 jwd joined
19:17 jri joined
19:23 silenced joined
19:24 arnsa joined
19:25 re1 joined
19:26 philipballew joined
19:27 re1 joined
19:37 harry1 joined
19:38 Muchoz joined
19:40 re1_ joined
19:43 harttho joined
19:46 fuzzybear3965 left
19:51 gentunian joined
20:12 orbyt_ joined
21:11 felixjet joined
21:20 druonysus joined
21:25 StephenLynx joined
21:34 Muchoz_ joined
21:34 Muchoz_ joined
21:36 philipballew joined
21:37 SkyRocknRoll joined
21:40 StephenLynx joined
21:46 culthero joined
22:14 MiquelEsco joined
22:14 MiquelEsco left
22:14 MicronXD joined
22:15 nemo_ joined
22:21 <MicronXD> given query results like: { "_id": "123", "tags": ["a","b","c"] } from a query like: { "tags": { "$in": ["a","b"] } }
22:22 <MicronXD> I understand I can project like so: { "tags" : { "$elemMatch": { "$in": ["a","b"] } } }
22:22 <MicronXD> but is there any way to make a new field called "matchedTags" instead of overwriting "tags"?
22:34 philipballew joined
22:44 DyanneNova joined
23:24 synchroack_ joined
23:27 kyuwonchoi joined
23:34 philipballew joined