When we all got locked down, I needed a project to keep my brain active. (That gets important as you get older: I just had my 80th birthday). My wife is learning Spanish and I tried writing a novel (it didn’t fly). In the end I decided to do something that I was good at – software development. I wrote a generic system to update and query a database using node.js. Test system here: http://www.sudsjs.com.
The database is accessed through a driver, which initially was written and tested for MySQL, PostgreSQL and SQLite 3. Since then, I have implemented (or tried) various NOSQL systems. So here is a summary and a link to any blog posts about all of the databases I have used.
The sales manager came down to the IT department and looked around. He spotted a young developer who looked a bit bored. His name plate said ‘Luke’. “I have a small job for you if you are interested”. Luke nodded. “We get sales orders from out customers. The warehouse handles them but we can’t get access to the data. Can you write a little program to store then on a computer for me.” Luke looked at a file of sample orders. “Sure no problem.”
He turned to the jaded old database administrator (Joda) in the corner “This is a great application for a NOSQL Document database like MongoDB or CouchDB.” Joda looked up – “That is a decision, regret you will”. Luke muttered something about ‘luddite’ and went back to his desk and coded up one NOSQL document per sales order, with customer details and an embedded array of order lines. The logical way to solve the problem.
I learned my software skills on the Ferranti Mercury computer. It had as I recall 468 words of memory (a word is about 6 bytes so call it 3k bytes). Let’s see, my phone has 8,000,000,000 bytes – that’s about 2.5 million times more. My phone makes the 1500 instructions per second look pretty silly as well.
The computer was big. The electronics used valves and they get hot. So each cabinet has a refrigerating unit built-in, and they were noisy.
My COVID lockdown project is a system to keep a database updated called SUDSJS. I am using it to check out different database management systems (DBMS). The latest is CouchDB – and I like it
I decided to use my lockdown project SUDSJS.com to test out a few different NOSQL database management systems (DBMS).
SUDSJS is written in JavaScript using the node.js server-side system and I designed it so that I could plug in different drivers for different databases. I wrote database drivers for MySQL, SQLite 3 and PostgreSQL. The first NOSQL system I tried was MongoDB, and the SUDSJS.com test site is currently running using this (correction – it is now running on CouchDB). The second NOSQL system I tried was Firestore from Google.
For the last 50 years there has been one and only one way of designing a database. SQL Database management systems (DBMS) have been the uncontested standard. (A better name is ‘relational’ because SQL is just the name of the language used to access them, but we will stick with it.) But things have changed in the last 10-15 years. Databases that disobey relational rules are coming into vogue. The term NOSQL is used for these non-relational databases (the above chart is work in progress, but I have been looking at these systems.) What is the difference between classic databases and this new breed?
You know what an invoice looks like. We have all seen thousands of them – probably too many. But invoices illustrate an interesting problem – and I am not talking about your VAT return. I am talking about the new kids on the database block – document databases. An example is MongoDB. These can store all the data for one invoice in a single structured record (called a ‘document’).
I am getting to the end of stage one in writing a database driver for MongoDB to use in my Lockdown Project sudsjs.com. All has gone well until I get to totalling up a field in the database. I had to enter the wonderful world of aggregation, which is powerful function-rich and confusing.
I was looking at a page on the IBM website last week when something odd happened. Half the page loaded, then there was a spinning circle (a.k.a. a throbber) for a second or so, then the rest of the page loaded. The page itself is a few paragraphs of very boring text plus a menu. So why the drama with the throbber for a page like that?
NOSQL databases like MongoDB are making a big splash in the computer industry. But are they set to replace the tried and tested technology of the past fifty years?
Here is my take on it, with a bit of history thrown in.
Bob Browning's blog 