Data modeling and organizational dynamics
Earlier this week I noted that, in choosing what aspect of a system to reason about first, engineering wisdom and politics often come apart. It's most often correct to figure out your data model first, but organizational and social pressures will work against you:
- Project-planning templates will often ask you about tools and subsystems; they'll rarely ask you about data structures.
- Managers and peers will tend to ask you what language or framework you're using; they're much less likely to ask you about which objects have which attributes.
- If you show too much interest in data-modeling questions, you often risk being perceived as a low-level, immature engineer.
Why are the organizational dynamics like this? Some candidate explanations:
- I am wrong, and they are right, and choosing the right programming language really is more important.
- Many people believe that data modeling decisions are independent of other system design decisions and that they matter only at an unimportant surface level.
- Tools and languages are more institutionally legible than data models: they are, for example, more likely to be things they (directly) pay for or pay to maintain.
- Questions like "what object should carry information about X?" don't often come up in coursework.
- Getting a data model right is often metaphysical in the sense that you need to figure out the structure of real-world things. Metaphysics is unpopular, so this aspect of data modeling is unpopular also.
I don't think this list is exhaustive, but I think some of those would figure importantly in a good explanation of what's going on.
This is all related to the problem of rewarding simplicity. Implementing simple designs and getting your data representations correct are important to being a good engineer but hard for institutions to reward or even understand (or, at least, to understand directly). If I'm right about this, improving at the craft will sometimes mean studying things that are hard to make intelligible to an organization.