Search This Blog

Showing posts with label week2. Show all posts
Showing posts with label week2. Show all posts

Thursday, December 26, 2013

Pymongo code example to manipulate data

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

In the various files in week2 directory you are going to find extensive code examples how to use the pymongo module. Below is a quick and short summary of the most important functions. I hope all code is self explanatory.

query = {'type':'exam', 'score':{'$gt':50, '$lt':70}}
iter = scores.find(query)

query = {'type':'exam'}
selector = {'student_id':1, '_id':0}
iter = scores.find(query, selector)

query = {'student_id':10}
doc = scores.find_one(query)

counter = counters.find_and_modify(query={'type':name},
update={'$inc':{'value':1}},
upsert=True, new=True)

cursor = scores.find(query).limit(10).skip(30)

query = {'media.oembed.type':'video'}
projection = {'media.oembed.url':1, '_id':0}
iter = scores.find(query, projection)

doc = {"name":"Andrew Erlichson", "company":"10gen","interests":['running', 'cycling', 'photography']}
people.insert(doc)

things.update({'thing':'apple'}, {'$set':{'color':'red'}}, upsert=True)
things.update({'thing':'pear'}, {'color':'green'}, upsert=True)

scores.update({},{'$unset':{'review_date':1}},multi=True)

scores.find_one({'student_id':1, 'type':'homework'})
score['review_date'] = datetime.datetime.utcnow()
scores.save(score)

scores.update({'student_id':1, 'type':'homework'},
{'$set':{'review_date':datetime.datetime.utcnow()}})

cursor = cursor.sort([('student_id',pymongo.ASCENDING),('score',pymongo.DESCENDING)])

Wednesday, December 25, 2013

How to find out how many documents were affected by the last instruction

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

getLastError attribute

On mongo shell the data modification operation not always generate any readable output to show what happened. The below trick can be used to:
  • Find last operation status (success or failure).
  • Find number of rows affected or changed ( the 'n' parameter )
  • To find out if a new document was inserted during insert-update
  • To find out how many documents were updated
 > db.people.insert( { _id : "top", var : 2} )
E11000 duplicate key error index: students.people.$_id_  dup key: { : "top" }
> db.runCommand( {getLastError : 1 } )
{
        "err" : "E11000 duplicate key error index: students.people.$_id_  dup key: { : \"top\" }",
        "code" : 11000,
        "n" : 0,
        "connectionId" : 1,
        "ok" : 1
}

---

> db.people.insert( { _id : "mark", var : 3} )
> db.runCommand( {getLastError : 1 } )
{ "n" : 0, "connectionId" : 1, "err" : null, "ok" : 1 }

---

> db.people.update( {var : { $lt:3 } } , { $set : { var2: 0 } }, {multi: true} )
> db.runCommand( {getLastError : 1 } )
{
        "updatedExisting" : true,
        "n" : 2,
        "connectionId" : 2,
        "err" : null,
        "ok" : 1
}
> db.people.find()
{ "_id" : "mark", "var" : 3 }
{ "_id" : "rado", "var" : 1, "var2" : 0 }
{ "_id" : "top", "var" : 2, "var2" : 0 }

---

# showing bad example with wrong order of functions! 

> db.people.update( {var : 4} , { $set : { var2: 0 } }, { upsert: true } )
> db.people.find()
{ "_id" : "mark", "var" : 3 }
{ "_id" : "rado", "var" : 1, "var2" : 0 }
{ "_id" : "top", "var" : 2, "var2" : 0 }
{ "_id" : ObjectId("52ba26f9c1207a4a1fdeb3ae"), "var" : 4, "var2" : 0 }
> db.runCommand( {getLastError : 1 } )
{ "n" : 0, "connectionId" : 2, "err" : null, "ok" : 1 }

---

# the same operation but with correct functions order 

> db.people.update( {var : 4} , { $set : { var2: 0 } }, { upsert: true } )
> db.runCommand( {getLastError : 1 } )
{
        "updatedExisting" : false,
        "upserted" : ObjectId("52ba2916c1207a4a1fdeb3b0"),
        "n" : 1,
        "connectionId" : 2,
        "err" : null,
        "ok" : 1
}
> db.people.find()
{ "_id" : "mark", "var" : 3 }
{ "_id" : "rado", "var" : 1, "var2" : 0 }
{ "_id" : "top", "var" : 2, "var2" : 0 }
{ "_id" : ObjectId("52ba2916c1207a4a1fdeb3b0"), "var" : 4, "var2" : 0 }

---

> db.people.update( {} , { $set : { var2: 1 } }, { multi: true } )
> db.runCommand( {getLastError : 1 } )
{
        "updatedExisting" : true,
        "n" : 4,
        "connectionId" : 2,
        "err" : null,
        "ok" : 1
}
> db.people.find()
{ "_id" : "rado", "var" : 1, "var2" : 1 }
{ "_id" : "top", "var" : 2, "var2" : 1 }
{ "_id" : ObjectId("52ba26f9c1207a4a1fdeb3ae"), "var" : 4, "var2" : 1 }
{ "_id" : "mark", "var" : 3, "var2" : 1 }

Updating _ID document attribute in MongoDB

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

The _id in MongoDB has a special meaning and is always present in every document. It is used as the primary key for the collection.

It doesn't have to be generated automatically. You can specify its value manually if you want.
 
> db.people.insert( { _id : "rado", var : 1} )
> db.people.find()
{ "_id" : "rado", "var" : 1 }

> db.people.insert( { _id : "top", var : 2} )
> db.people.find()
{ "_id" : "rado", "var" : 1 }
{ "_id" : "top", "var" : 2 }

> db.people.insert( { _id : "top", var : 2} )
E11000 duplicate key error index: students.people.$_id_  dup key: { : "top" }

> db.people.find()
{ "_id" : "rado", "var" : 1 }
{ "_id" : "top", "var" : 2 }

Removing data

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course
 
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 2, "tab" : [  11,  22,  "hello" ] }
{ "_id" : ObjectId("52ba1e4cc1207a4a1fdeb3ac"), "a" : 3, "tab" : { "2" : "london" } }
{ "_id" : ObjectId("52ba1e99c1207a4a1fdeb3ad"), "a" : 4, "mystr" : "poland" }
>
> db.arrays.remove( { a: 2})
> db.arrays.find()
{ "_id" : ObjectId("52ba1e4cc1207a4a1fdeb3ac"), "a" : 3, "tab" : { "2" : "london" } }
{ "_id" : ObjectId("52ba1e99c1207a4a1fdeb3ad"), "a" : 4, "mystr" : "poland" }

This removes all the documents in the collection at once. The drop method is more recommended because it drops the whole collection instead or removing document by document (more performant).
 
> db.arrays.remove()
> db.arrays.drop()
> db.arrays.find()

IMPORTANT: multi document operation are not atomic and the db engine execution thread after removing a few documents can yield the execution to another thread before returning and finishing its operation.

Update on multiple records inside collection

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

By default an update function with 2 arguments updates a SINGLE RANDOM (unspecified) document. To update multiple documents we need to provide the 3th option: multi.

The {} as first argument means match every document.
 
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 1, "tab" : [  11,  22,  "hello" ] }
{ "_id" : ObjectId("52ba1e4cc1207a4a1fdeb3ac"), "a" : 2, "tab" : { "2" : "london" } }
{ "_id" : ObjectId("52ba1e99c1207a4a1fdeb3ad"), "a" : 3, "mystr" : "poland" }

> db.arrays.update( { }, { $inc : { "a" : 1 } }, { multi: true} )
>
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 2, "tab" : [  11,  22,  "hello" ] }
{ "_id" : ObjectId("52ba1e4cc1207a4a1fdeb3ac"), "a" : 3, "tab" : { "2" : "london" } }
{ "_id" : ObjectId("52ba1e99c1207a4a1fdeb3ad"), "a" : 4, "mystr" : "poland" }

IMPORTANT: MongoDB doesn't offer isolated transactions across multiple documents. It grantees a single document update is atomic. No other concurrent updates/writes can modify the same document.

In regards to the test above it means that there is no grantee that all the updates will be executed in one go.

It is possible that the db engine thread that does the update will be stopped, de-scheduled or its execution will be yielded (will be give) to another db engine thread that modifies the same document collection (or its subset). In such a case the updates on the documents may happen in different orders and one can override the other. Only a single document update is atomic.

Insert and update operation

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

Update or insert a new record into a document if it doesn't exist.
 
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 1, "tab" : [  11,  22,  "hello" ] }

> db.arrays.update( { a: 2}, { $set : { "tab.2" : "london" } } )
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 1, "tab" : [  11,  22,  "hello" ] }


> db.arrays.update( { a: 2}, { $set : { "tab.2" : "london" } }, { upsert: true} )
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 1, "tab" : [  11,  22,  "hello" ] }
{ "_id" : ObjectId("52ba1e4cc1207a4a1fdeb3ac"), "a" : 2, "tab" : { "2" : "london" } }

> db.arrays.update( { a: 3}, { $set : { "mystr" : "poland" } }, { upsert: true} )
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 1, "tab" : [  11,  22,  "hello" ] }
{ "_id" : ObjectId("52ba1e4cc1207a4a1fdeb3ac"), "a" : 2, "tab" : { "2" : "london" } }
{ "_id" : ObjectId("52ba1e99c1207a4a1fdeb3ad"), "a" : 3, "mystr" : "poland" }

Modifying documents in the data base

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

Document updates

The update syntax is similar to how find function works. The first argument is a search document, the second is the NEW DOCUMENT to be inserted to REPLACE the existing.
 
> db.grades.find( { "student_id" : 1})
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "student_id" : 1, "type" : "quiz", "score" : 96.76851542258362 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57d"), "student_id" : 1, "type" : "homework", "score" : 21.33260810416115 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57e"), "student_id" : 1, "type" : "homework", "score" : 44.31667452616328 }
>
> db.grades.update( { "student_id" : 1}, { myvar : "new doc2"} )
>

> db.grades.find( { $or : [ { "student_id" : 1 }, { myvar : "new doc2"} ] } )
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "myvar" : "new doc2" }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57d"), "student_id" : 1, "type" : "homework", "score" : 21.33260810416115 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57e"), "student_id" : 1, "type" : "homework", "score" : 44.31667452616328 }

Adding or modifying new attribute in the document
 
> db.grades.find( { myvar : "new doc2"} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "myvar" : "new doc2" }
>
> db.grades.update( { myvar : "new doc2"}, { $set : { newvar : 1} } )
>
> db.grades.find( { myvar : "new doc2"} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "myvar" : "new doc2", "newvar" : 1 }

You can increase an existing attribute or add a new one if it doesn't.
 
> db.grades.update( { myvar : "new doc2"}, { $inc : { newvar : 100} } )
>
> db.grades.find( { myvar : "new doc2"} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "myvar" : "new doc2", "newvar" : 101 }
>
> db.grades.update( { myvar : "new doc2"}, { $inc : { newnumber : 2} } )
>
> db.grades.find( { myvar : "new doc2"} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "myvar" : "new doc2", "newnumber" : 2, "newvar" : 101 }

As MongoDB don't have a strict db schema we can dynamically remove an attribute as well.
 
> db.grades.update( { myvar : "new doc2"}, { $unset : { newnumber : 2} } )
> db.grades.find( { myvar : "new doc2"} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "myvar" : "new doc2", "newvar" : 101 }

If a value is an array you can update a individual element of the array in the document.
 
> db.arrays.insert( { a : 1, tab : [ 11,22,33] } )
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 1, "tab" : [  11,  22,  33 ] }
>

> db.arrays.update( { a: 1}, { $set : { "tab.2" : "hello" } } )
> db.arrays.find()
{ "_id" : ObjectId("52ba1c78a83c1ee5e6c903a1"), "a" : 1, "tab" : [  11,  22,  "hello" ] }

There are other methods you can use. A complete list of them can be found at http://docs.mongodb.org/manual/reference/operator/update-array

How to count documents in collection

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

The count method allows you to find the number of documents in your collection.
 
> db.grades.count()
809

> c = db.grades.find(); c.limit(3)
{ "_id" : ObjectId("50906d7fa3c412bb040eb577"), "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb578"), "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb579"), "student_id" : 0, "type" : "homework", "score" : 14.8504576811645 }

> c.count()
809

Tuesday, December 24, 2013

Cursors in MongoDB and mongo shell Javascrip API

test Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

The mongo shell support cursors. There are many functions available with this object.
 
> mycursor = db.grades.find(); null

> mycursor.
mycursor.addOption(             mycursor.forEach(               mycursor.max(                   mycursor.showDiskLoc(
mycursor.arrayAccess(           mycursor.hasNext(               mycursor.min(                   mycursor.size(
mycursor.batchSize(             mycursor.hasOwnProperty(        mycursor.next(                  mycursor.skip(
mycursor.clone(                 mycursor.help(                  mycursor.objsLeftInBatch(       mycursor.snapshot(
mycursor.comment(               mycursor.hint(                  mycursor.pretty(                mycursor.sort(
mycursor.constructor            mycursor.itcount(               mycursor.propertyIsEnumerable(  mycursor.toArray(
mycursor.count(                 mycursor.length(                mycursor.readOnly(              mycursor.toLocaleString(
mycursor.countReturn(           mycursor.limit(                 mycursor.readPref(              mycursor.toString(
mycursor.explain(               mycursor.map(                   mycursor.shellPrint(            mycursor.valueOf(

> mycursor.hasNext()
true

By using the mongo shell flexibility you can run JavaScript code. The simple program below retrieves and prints values from the collection as demonstration:
 
> for ( var i =1 ; i<5 && mycursor.hasNext(); i++ ) { print("for iteration # " + i); printjson(mycursor.next()) }
for iteration # 1
{
        "_id" : ObjectId("50906d7fa3c412bb040eb5b4"),
        "student_id" : 15,
        "type" : "quiz",
        "score" : 33.87245622400884
}
for iteration # 2
{
        "_id" : ObjectId("50906d7fa3c412bb040eb5b5"),
        "student_id" : 15,
        "type" : "homework",
        "score" : 18.41724574382455
}
for iteration # 3
{
        "_id" : ObjectId("50906d7fa3c412bb040eb5b6"),
        "student_id" : 15,
        "type" : "homework",
        "score" : 7.475648374118382
}
for iteration # 4
{
        "_id" : ObjectId("50906d7fa3c412bb040eb5b7"),
        "student_id" : 16,
        "type" : "exam",
        "score" : 40.92812784954744
}

There is a way to limit the number or documents to read as well.
 
> mycursor = db.grades.find(); null
null
> mycursor.limit(5); null
null
> while ( mycursor.hasNext()) { printjson(mycursor.next()) }

We can apply a sort function to the cursor as well. The code shows how to list documents in the revers order.
 
> mycursor = db.grades.find(); null
null
> mycursor.limit(5); null
null
> mycursor.sort( { student_id : -1 } )
{ "_id" : ObjectId("50906d7fa3c412bb040eb893"), "student_id" : 199, "type" : "exam", "score" : 67.33828604577803 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb894"), "student_id" : 199, "type" : "quiz", "score" : 48.15737364405101 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb895"), "student_id" : 199, "type" : "homework", "score" : 49.34223066136407 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb896"), "student_id" : 199, "type" : "homework", "score" : 58.09608083191365 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb88f"), "student_id" : 198, "type" : "exam", "score" : 49.65504121659061 }

In the code above this is happening:
  • Create a cursor - no data has been received from the db at this time; merely we have established a db connection with the collection.
  • We say we want to see only 5 documents.
  • When we execute the sort method it fetches data from db, iterates over the cursor and print 5 documents (notice the missing null at the end)
And even further we can concatenate the two functions together on a single line.
 
> mycursor.sort( { student_id : -1 } ).limit(5)
{ "_id" : ObjectId("50906d7fa3c412bb040eb893"), "student_id" : 199, "type" : "exam", "score" : 67.33828604577803 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb894"), "student_id" : 199, "type" : "quiz", "score" : 48.15737364405101 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb895"), "student_id" : 199, "type" : "homework", "score" : 49.34223066136407 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb896"), "student_id" : 199, "type" : "homework", "score" : 58.09608083191365 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb88f"), "student_id" : 198, "type" : "exam", "score" : 49.65504121659061 }

The take away from this is examples is to note that all the function applied to the cursor are not processed on the client site by the mango shell but instead these are instruction sent to the MongoDB engine once the cursor is created and being processed by the engine once the cursor request data to be returned.

The cursor is an interface to sent the parameters to the db engine and to read the returned data.

Nested documents and the dot notation in find

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

Nested documents

We can have a variable that represent another document with completely separate attributes and values.
 
> db.embeded.insert( {rado1:1, nestedDoc : { var1 : 1, str : "test"  } } )

> db.embeded.find()
{ "_id" : ObjectId("52b8e2aa8a25e81047ccb8c9"), "rado1" : 1, "nestedDoc" : { "var1" : 1, "str" : "test" } }
> db.embeded.find().pretty()
{
        "_id" : ObjectId("52b8e2aa8a25e81047ccb8c9"),
        "rado1" : 1,
        "nestedDoc" : {
                "var1" : 1,
                "str" : "test"
        }
}

Searching for embedded variables

This is the find criteria for the most outer document variable in the standard way we were doing so far.
 
> db.embeded.find( {rado1 : 1} )
{ "_id" : ObjectId("52b8e2aa8a25e81047ccb8c9"), "rado1" : 1, "nestedDoc" : { "var1" : 1, "str" : "test" } }

Taking this analogy this search unfortunately doesn't produce the expected result.
 
> db.embeded.find( {nestedDoc : { var1 :1 } } )
>

If you need to provide the argument for the find method that is an embedded documents you need to provide the exact variables it is built of. Pay attention to the order because if you mix it this will not work again. It doesn't work because MongoDB is looking for a variable with exactly this schema/syntax specified.
 
> db.embeded.find( {nestedDoc : { var1 :1, str: "test" } } )
{ "_id" : ObjectId("52b8e2aa8a25e81047ccb8c9"), "rado1" : 1, "nestedDoc" : { "var1" : 1, "str" : "test" } }
> db.embeded.find( {nestedDoc : { str: "test", var1 :1 } } )

Dot notation

To search for the embedded values we can use the "dot" notation.
 
> db.embeded.find( { "nestedDoc.var1" : 1 } )
{ "_id" : ObjectId("52b8e2aa8a25e81047ccb8c9"), "rado1" : 1, "nestedDoc" : { "var1" : 1, "str" : "test" } }

Monday, December 23, 2013

Document attribute that is an array of strings

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

Arrays

When a document attribute is an array MongoDB searches through the array values when you use the find() function. This can be odd if you are not aware of such behavior. Even though this feature is important to note that there is not support for recursive searches on array values as such.

If the attribute value is an array the below query will perform the matching against the array elements as well.
 
> db.grades.find( { "test" : 2,  "test var" : "london" } )
{ "_id" : ObjectId("52b78121e7a1580719615d4f"), "test" : 2, "test var" : [  "munich",  "london" ] }
{ "_id" : ObjectId("52b78125e7a1580719615d50"), "test" : 2, "test var" : "london" }

For more advance searches you can the $all and $or with arrays.
The $all operator means that your attribute array needs to have all the specified values.
 
> db.grades.find( { "test" : 2,  "test var" : { $all : [ "london" , "munich"]  } } )
{ "_id" : ObjectId("52b78121e7a1580719615d4f"), "test" : 2, "test var" : [  "munich",  "london" ] }

Where the $in operator means that your document array attribute needs to have at least one of the specified strings.
 
> db.grades.find( { "test" : 2,  "test var" : { $in : [ "london" , "munich"]  } } )
{ "_id" : ObjectId("52b78121e7a1580719615d4f"), "test" : 2, "test var" : [  "munich",  "london" ] }
{ "_id" : ObjectId("52b78125e7a1580719615d50"), "test" : 2, "test var" : "london" }

The same document attribute can have different type in a single collection

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course


It is possible that multiple documents in a single collection have the same attribute of different type. In the example below the test attribute has a value number 1 on the first document and a string value in the second.
 
> db.grades.insert( {"test" : 1, "test var" : 1 } )
> db.grades.insert( {"test" : 1, "test var" : "string one" } )
>
> db.grades.find( { "test" : 1 })
{ "_id" : ObjectId("52b77870e7a1580719615d4a"), "test" : 1, "test var" : 1 }
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }

Pretty printing

The default formatting can be change to improve readability of some documents when applying the pretty() function.
 
> db.grades.find( { _id: ObjectId("50906d7fa3c412bb040eb577" ) } )
{ "_id" : ObjectId("50906d7fa3c412bb040eb577"), "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
>
> db.grades.find( { _id: ObjectId("50906d7fa3c412bb040eb577" ) } ).pretty()
{
        "_id" : ObjectId("50906d7fa3c412bb040eb577"),
        "student_id" : 0,
        "type" : "exam",
        "score" : 54.6535436362647
}

More advance search queries

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

Operators in search queries

We can ask for values greater than.
 
> db.grades.find( {"student_id" : { $gt: 0, $lt:3 }, "type" : "quiz"} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "student_id" : 1, "type" : "quiz", "score" : 96.76851542258362 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb580"), "student_id" : 2, "type" : "quiz", "score" : 1.528220212203968 }

We use the exists operator when we want to retrieve documents with specific attribute regardless of its value or type. The only condition is that this attribute needs to exists in the document.
 
> db.grades.insert( {"test" : 1, "test var" : "string one", "other var" : 2 } )
> db.grades.find( { "other var" : {$exists : true}  })
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

> db.grades.find( { "test var" : {$exists : true} })
{ "_id" : ObjectId("52b77870e7a1580719615d4a"), "test" : 1, "test var" : 1 }
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

Finding documents with attribute of a particular type.
 
> db.grades.find( { "test var" : {$type : 1} })
{ "_id" : ObjectId("52b77870e7a1580719615d4a"), "test" : 1, "test var" : 1 }

> db.grades.find( { "test var" : {$type : 2} })
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

You can use as well as regular expression when matching document attributes.
 
> db.grades.find( { "test var" : {$regex : "^s.*one$" }}  )
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

If you want to provide 2 separate searching criterias you can use the $or operator.
 
> db.grades.find( { $or : [{ "test var" : {$type :1 }}, {"test var" : {$regex : "^s.*one$" }}] } )
{ "_id" : ObjectId("52b77870e7a1580719615d4a"), "test" : 1, "test var" : 1 }
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

In a similar way the $and will help you to write queries where a single attribute needs to meet multiple criterias.
 
> db.grades.find( { $and : [ {"test var" : {$regex : "^s" }}, { "test var" : { $regex : "one$"} }] } )
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

The $and queries can be further optimized and rewrite to use simpler syntax.
 
> db.grades.find( { "test var" : {$regex : "^s", $type : 1} } )
>
> db.grades.find( { "test var" : {$regex : "^s", $type : 2} } )
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

It is important how do you write the search criterias above. If you write them separately you will create different query. The attribute criteria may be overwritten and only the last one will be used.
 
> db.grades.find( { "test var" : {$regex : "^s" }, "test var" : { $type : 1} } )
{ "_id" : ObjectId("52b77870e7a1580719615d4a"), "test" : 1, "test var" : 1 }

> db.grades.find( { "test var" : {$regex : "^rrrrrr" }, "test var" : { $type : 2} } )
{ "_id" : ObjectId("52b77877e7a1580719615d4b"), "test" : 1, "test var" : "string one" }
{ "_id" : ObjectId("52b7791fe7a1580719615d4c"), "test" : 1, "test var" : "string one", "other var" : 2 }

How to search for documents in collection

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

In RDMS to look for what data are stored in a table we would use SELECT. In MongoDB we are going to use find method instead.

Search method

For the search function you provide 2 documents as arguments.
The 1st arg is what you are looking for.
The 2th what values do you want to retrieve.
 
> db.funnynumbers.insert( { rado: 1, a:100, b:200 , c:300} )
>
> db.funnynumbers.find( {a:100} )
{ "_id" : ObjectId("52b252ff95aed72de4e6ed39"), "rado" : 1, "a" : 100, "b" : 200, "c" : 300 }
>
> db.funnynumbers.find( {a:100}, {rado:true} )
{ "_id" : ObjectId("52b252ff95aed72de4e6ed39"), "rado" : 1 }
> db.funnynumbers.find( {a:100}, {rado:true, _id : false} )
{ "rado" : 1 }

Another example.
 
> db.grades.find()
{ "_id" : ObjectId("50906d7fa3c412bb040eb577"), "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb578"), "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb579"), "student_id" : 0, "type" : "homework", "score" : 14.8504576811645 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57a"), "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57b"), "student_id" : 1, "type" : "exam", "score" : 74.20010837299897 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57c"), "student_id" : 1, "type" : "quiz", "score" : 96.76851542258362 }

You can specify multiple search criterias by specifying multiple document attributes.
 
> db.grades.find( {"student_id" : 0} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb577"), "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb578"), "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb579"), "student_id" : 0, "type" : "homework", "score" : 14.8504576811645 }
{ "_id" : ObjectId("50906d7fa3c412bb040eb57a"), "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }

> db.grades.find( {"student_id" : 0, "type" : "quiz"} )
{ "_id" : ObjectId("50906d7fa3c412bb040eb578"), "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }

Mongo shell

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

The interactive shell is called mongo. It is JavaScript interpreter. Everything it interprets is a JavaScript code.

Default variables
 
$ mongo
MongoDB shell version: 2.4.8
connecting to: test
> db
test
> show dbs
local   0.078125GB
m101    0.203125GB
test    (empty)
> use m101
switched to db m101
> db
m101

You can even run some JavaScript code to implement more sophisticated behavior wehn interacting with data base.
 
> for ( var i =1 ; i<5; i++ ) { print(i) }
1
2
3
4

_ID document attribute

Every retrieved document from database has an attribute '_id'.
 
> db.funnynumbers.findOne()
{ "_id" : ObjectId("50778ce69331a280cf4bcf7d"), "value" : 87 }

It is used by the MongoDB engine internally. It is used as a PRIMARY KEY for collection for example.

The _id is immutable. To change it you can remove and insert a document again into db.

The _id is dynamically calculated when you insert documents into db. To grantee that there are no collision its value will be based on current time and process id.
 
> db.funnynumbers.findOne()
{ "_id" : ObjectId("50778ce69331a280cf4bcf7d"), "value" : 87 }


> db.funnynumbers.find( {'rado' : 1} )
{ "_id" : ObjectId("52b251f995aed72de4e6ed38"), "rado" : 1 }

JavaScript script to populate the data base

In the week2 we can find a JavaScript code that uses the power and flexibility of the mongo shell to creates the collection and insert documents:

https://github.com/rtomaszewski/mongo-course/blob/master/M101P/week2/create_student_collection.d59b66847ae9.js
 
~/mongo-course/M101P/week2# mongo < create_student_collection.d59b66847ae9.js
MongoDB shell version: 2.4.8
connecting to: test
switched to db school
bye

References

http://docs.mongodb.org/manual/reference/mongo-shell/

MongoDB supported features

Important note: This article is in relation to online MongoDB course. For more information about the course and other posts describing its content please check my main page here: M101P: MongoDB for Developers course

What is MongoDB data base and what does it support
  • Is a document oriented data base.
  • Has a dynamic schema (db schema can be created dynamically when you run quires, insert or updates documents in collection).
  • Document access is guaranteed to be atomic. Supports atomic operation on a single document. There is no support for transaction on multiple documents or documents across multiple collections.
  • It supports Indexes and Secondary Indexes.
  • Doesn't use SQL language like the one use for RDMS. Instead it uses an API like language for data manipulation and quires.
  • Interestingly the mongo shell is an interactive JavaScript interpreter. 
  • It uses JSON as a document format when interacting with external world.
  • Documents and data base data are stored in a binary BSON format on the disk by the engine although.
  • Data normalization is not a primary goal for document stored in MongoDB. Instead of trying to normalize all data stored in collection in Mongo you are advise to use embedding for 1-1, 1-many, many-1 and many-many relationships. Of course if this is not aligned with your application data access path you can still normalize data and use the _ID as a reference across tables. But this will be client site to enforce this to keep the data consistent. 
  • Support single and multi key indexes.
  • A single file you can save can be up to 16MB in size. To store larger files you can use gridfs.


What is not supported in MongoDB
  • Doesn't' supports joins.
  • Doesn't support RDMS SQL language.
  • Doesn't support transactions across multiple collections.
  • Doesn't support transactions across multiple documents.
  • Doesn't natively support constrains (example are foreign keys in RDMS tables). If you need a mechanism like this you would need to enforce it on the client site. Often you can lower the requirements for this feature when using a smart documents schema with embedded documents.