Connect with me:

Blog Posts

Pervasive Myths, Databases and Development – A Series

Part One: The Impedance Mismatch is Real !! (not)

I’ve been waiting for about 10 years to write this series. If you’re a software developer and you disagree with me that is fine. All I ask is that you turn the other cheek. If you are a CTO, Development Manager, Architect, or Technical Lead pay close attention, withhold judgment until the end. This is only part one, there is more to follow at a later time (this article does not address the code first approach, that is addressed later in this series).

I’d be walking you to the door …

damnyouYou can stomp and up and down and yell all you want Damn You !! The Impedance Mismatch is real, its REAL!! and that one’s object model should not look like the database schema and I say you’re simply full of crap (sorry for the tone but it truly is warranted here). Were I your manager I would kindly walk you to the door before allowing you to waste six months or more of my companies budget and maybe even put me out of business. As you see security bearing down on you, in a panic, you might even try to pull up a Wikipedia article and say “See, see I told you so …” but we’d still be headed toward the door.

The risk in believing the myth …

I have been developing applications and architectures on top of databases for well over 20+ years and have never, ever run into this so called “Impedance Mismatch” problem. Nothing I have ever faced has caused me to design a set of objects that is modeled entirely different than my database or even differently at all for that matter.

When I am a consultant I often see projects that are in real deep trouble, and many of them are brand spanking new rewrites (too many in fact). Most often the reason is the development team or team lead somehow believes this myth that their object model and their database are entirely different “things”, have different purposes, and serve different needs. I however, say exactly the opposite, the two are in fact one and the same.

So, believing this myth what does your development team do? They set off on a grand adventure to create this mythological object model thereby wasting your money and costing valuable development time. The danger here is twofold. First, your development team is wasting a lot of time and money and well on their way to creating what is sure to be a maintenance nightmare. Second, your biggest competitor might have me as their Architect and I assure you I am not wasting your competitor’s money nor squandering their time.

A good Architect is not enamored with technology such that he wants to play with it, nor experiment with it on his employer’s dime when he knows there is a sure fire proven way to accomplish something. This would be to gamble with the companies future and the future of all the employees who depend on it. A good Architect wants to get the project done and get it done fast and there is a sure fire way to accomplish this, and he knows how.

First, let’s take a look at what your development team is doing in middle earth and then we’ll take a look at what I am doing for your competitor.

What your development team is doing

Let’s assume for this exercise the database already exists.

The first and most critical mistake your development team makes is overlooking potentially the most valuable resource in your company, the database schema itself. The database schema contains all of the tables and views. Each table has a set of columns, each column has information such as its data type, primary keys, data length, default value, whether it’s required or not, and so much more. The table has a list of foreign keys that not only enforces but defines relationships between other tables. For example, an employee has a department id which is enforced by a foreign key between the Employee and Department table. There is still more valuable information such as indexes on tables indicating which fields are tuned for quick searches.

But rather than use this valuable information, which, by definition is correct, they scoff at it as if it were their enemy. In fact, they may even be falling for a more sinister myth such as relational databases are evil and quickly becoming a thing of the past. This sinister myth will be covered in an upcoming article in this series. Needless to say, relational databases will still be alive and well a hundred years from now. How do I know this? Because data is relational, it’s a fact that is never going away.

So, your development team has decided to ignore and therefore not utilize all of this incredibly valuable schema information describing your database. And your database consists of 250+ tables. Still, they have decided to go it alone. They’re going to hand code all this information. This alone will take probably months. There also has to be some kind of underlying architecture written to accompany it. Your development staff simply sees this as the cost of development, after all, the object model and database in their minds are two entirely different things. Of course, when you’re database schema changes from release to release they get to duplicate all these changes into their object model by hand as well.

Ultimately, your development staff has to now know and understand twice as much information because they chose to write this mythological object model. They have to understand both the database and the object model that they just spent two or three months developing, that is, if you’re lucky. I’ve seen teams waste as much as six months on such a layer. Don’t think for a moment that your developers won’t have to thoroughly understand the database also, they will. There are always complicated tasks that require sophisticated stored procedures and views that need to be created. At this point your development team has created abstract objects that do not map to a particular table or view. The names of these objects are not the names of tables and their properties are different than the column names they are eventually stored in. This makes it infinitely more difficult for your developers to write, understand, and debug. And trust me, it leads to confusion and bugs which is yet more wasted time and money wasted, and I mean utterly wasted.

Remember, almost everything we write as developers exists to eventually store information in a database. Why should it be such a confusing venture, honestly, why?

By now there is probably some grumbling by a few of the developers on the team who have realized the same thing. Why is this so damn confusing? You might even experience some turn over, which is doubly painful because your developers have created a system that is infinitely more complex (needlessly so) than the system that I am creating for your competitor. So your new hire learning curve is much more difficult than it should be which delays the time in which a new hire begins to be able to truly contribute to the team. More lost time and money.

Maybe now is the time you might want to ask your development team if they have documentation which describes this object model they have created? Do they have a document that maps their mythological middle earth properties to their hardened real database column counterparts? I’m guessing not, they’ve no time because they’re months behind schedule by now. So, your new hires have two sets of lingo to learn, the database and the object model, and they look completely different. In fact, the object model might not be very intuitive at all, with developers not even knowing when an object will hydrate (or load data from database) at all and have little control over that process. Even if your team uses a modeling tool most of the problems with this approach still exist.

Let’s stop here before I blow a gasket.

What I am doing for your competitor

I know the true value of my companies database schema, it’s priceless information. In fact, it is a specification within itself. Since I am familiar with EntitySpaces I am going to use it as my tool in this scenario. However, there are many other tools that can accomplish the same thing, the tool is 100% free so I am not trying to hock a product here, I’m trying to show you how I get done six months before your team.

The first thing I am going to do is install EntitySpaces, it takes about 5 minutes. Next, I am going to connect to my database and execute two templates. The first template is the “Generated” template. What does this template do? It reads all of the metadata from my database and generates a few classes for each table in C# or VB.NET.  For instance, if I have an Employee table then I will get these three classes:

Generated Classes

  1. esEmployee
  2. esEmployeeCollection
  3. esEmpoyeeQuery

Notice I don’t have to guess what table these classes read and write to, it’s obviously the “Employee” table. Next I generate the “Custom” Template, this creates these three classes:

Custom Classes

  1. Employee
  2. EmployeeCollection
  3. EmpoyeeQuery

Let me explain how these classes work. But, before I do let me make it clear that I generated all of the classes for all 250+ tables in my databases in less than 15 minutes. That right, 15 minutes. So, how do the Generated and Custom classes relate to each other? It’s simple, the Custom classes inherit from the Generated classes.

public class Employee : esEmployee {}
public class EmpoyeeCollection : esEmployeeCollection {}
public class EmployeeQuery : esEmployeeQuery {}

My team never hand changes the “Generated” classes. Instead, whenever our schema changes we simply regenerate the classes. Since “Custom” classes inherit from the “Generated” classes and allow me to override properties and methods I have full control. Plus I can write my business logic in the Custom classes. This is nice because my Custom classes contain only my business logic and are not cluttered with underlying architecture code. I only generate the Custom classes once and never again, I am free to modify them to my heart’s desire.

Active Record

This is the Active Record model and it works, in fact it works very well. Ruby on Rails uses the Active Record approach only it dynamically pulls the metadata from the database at runtime. Basically, my object model and my database look exactly the same, and they should. Remember that Employee / Department Id example we spoke of earlier? Well, my Employee object will have a Department property because EntitySpaces understands foreign keys and knows how to generate the code such that it looks exactly like the database. The full object model is expressed in the code and reflects the same exact relationships that are in my database.

Advantage Me

The developers on my team didn’t spend three months or more writing mind numbing, tedious code. Instead, we generated the code in 15 minutes. They also won’t have to waste time learning some abstract, mythological, object model that has no real reason to exist. If they understand the database then they understand the object model and vice versa. So, we already have our full, working object model which can read and write entire hierarchical objects graphs to and from the database using transactions. Meanwhile, your team lead is still in Outlook trying to figure out which conference room they can book for their kick off meeting in which their going to talk about this “cool” new object model they are going to write.

My team is going to take all that time and money your team is going to waste and we’re going to pour it into our user interface and additional features that your team will have ultimately have to cut. And since we’re competitors we’re going to end up with your customers. My development team is going to be happy because the bulk of the grunt work is done, and now they get to do the fun part, that is, write the actual application. In fact, what my company needs isn’t an object model, they need the application and they need it to perform well and operate as expected. Because we are using the active model our developers aren’t confused, they inherently understand the object model and their code looks like the problem they are trying to solve.

I know some will disagree with me on this, but when you’re told by companies like Ernst and Young and others that you’re incredibly fast and cheap it’s because you beat their schedules and can react to change with near immunity, simply regenerate. And the way you get those compliments is by taking approaches like the one mentioned above. Of course, there’s a little “tongue in cheek” on the walking to the door part, but I had to get you to read the whole article somehow ….

Leave a Reply